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ABSTRACT 

* The purposes of the project described in this report 
were to train curriculum specialists and to develop au>d test 
curriculum design training materials. The first phase of the project 
included^ the development of training materials and their use to train 
a small 'group of students. During the training of the pilot group of 
students, an effective design model was identified and a format 
developed for self-instructional materials. The design was a 
structured-curriculum, model supplemented .with additional elements and 
procedures suitable for complex subject matter,, called the process 
model for individualization of curricular (PIC). This model became 
the b^ses for design of the curriculum course, and the PIC procedures 
w^re taught* as a design model. This phase of the project ended with a 
revision of the materials based on the' experiences and^results 
encountered with the students.. The second phase of thSFprpject was a 
series of field tests of the materials and the^r revision based on 
formative evaluation procedures. (Author/HLF) 
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' CURRICULUM DESlGty AND DEVELOPMENT PROJECT 
Introduction 

The Curriculum Design and Development Project, conducted by the 
Learning Research and Development Center at the University of Pittsburgh 
in cooperation with the Department of Curriculum and Supervision at the 
University, had two purposes: to train curriculum specialists and to 
develop and test curriculum design trailing materials. There has been 
a shortage of personnel trained in the highly specialized procedures 
which have sprung from. the emerging field of instructional technology. 
Most research and development i centers and regional laboratories, there- 
fore, have had to provide on-the-job training for staff members of 
curriculum development projects. In addition, there is a need to dis- 
seminate to teachers and administrators training materials which can 
help them to evaluate and -select or develop their own individualized 
instructional materials. 

This project was planned in two phases. The first phase would 
include th^ development of training materials and their use to train a 
small group of students, who would also take other courses at the 
University, would serve internships on curriculum projiects, would write 
dissertations on curriculum-related topics and would receive doctorates 
from the Department of Curriculum and Supervision of the School of 
Education, University of Pittsburgh. The major pdrt ot their training 
in curriculum would corie from tlje materials the project would design 
and develop. This phase of the project would end with a revision of the 
materials based on the experiences and results encountered with the 
pilot group of s,tudent3. The second phase of the project w^uld be a 
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series of field tests of the materials and their revision based on 



formative evaluation procedures. ^ 

The intensive pilot- testing effprt was carried out during 1971 - 
72. Phase two of the project, the series of field tests, extended from 
Fall of 1972 to December, 1973, involving over one hundred students. 
The activity of these two phases is presented in the following report. 




PROGRAM DEVELOPMENT: PHASE 1 
The Training ^ 

The Curriculum Design and Development Project, in it initial phase, 
trained the pilot gfoup of students while the training materials were 
being developed , so ^that the most effective methods, strategies, pro- 
cedures and materials could be incorporated in the final product. The 
pilot students worked intensively for two terms with the Project staff, 
providing extensive feedback on the instructional materials, which 
underwent almost continuous revision. 

The original cup^icuTui^ design course included an appreciable 
amount of class discussion, a number of gue»€lectures, readings and 
worksheets, field trips, individual and groop_j?es^arch projects and one- 
to-one interaction with the course instructors. While each of these 
activitiesf^ceemed to provide useful insight or experience in some phase 
off curriculum design, the students requested more self-instructional 

materials. ^ ^ 

A change to a more completely self-instructional format would make 
the program product better suited to dissemination. At the same time, 
to accommodate to students with different backgrounds and different 
requirements there was a need to organize the instruction and to provide 
diagnostic testing for maximum flexibility in selecting objectives and 
progressing through the program. 

During the training oi the pilot group of students, an effective 
design model was identified and a format developed to meet these needs. 
The design, model was a structured-curriculum model, as described by 
Lindvall and Cox (1969), supplemented with additional elements and pro- 

3 



cedures suitable for complex subject matter, called the process model for 
individualization of curr-ieu^a (PIC). (See Appendix A.) This model ^ 
became the basis for design of the curriculum course and the PIC pro- 
cedures were taught as a design model. v 

Each unit of the self-instructional materials includes, a rationale 

0 

for selection of the unit objectives and their relation to the rest of 
the cours.e; a listing of the terminal and prerequisite objectives; and 
a charted hierarchy of the unit objectives, graphically demonstrating 
their interrelationships. One or more study guides incorporate tasks, 
which serve as curriculum embedded items to help the student monitor 
his progress. Reading sources are listed for each task and are coded 
to the unit objectivea.' Each unit includes an overview; articles from 
journals and Learning Research and Development Center publications; a 
bibliography; a posttest and answer keys. (See course materials and 
The Training Product). 

This first phase of the program proved ^to be an efficient way of 
developing training materials and, simultaneously, providing intensive 
training f^r a small group of students. Without the students it would 
not have been possible to , produce proven training materials expeditiously. 
At the same time, the concentrated training proved highly effective. 
This training has been evaluated by case studies of the original four 
doctoral students. These studies are elaborated here and in Appendices 
C and D. 

The Pilot Students 

The pilot students who are now completing their work for the doc- 
torate from the Department of Curriculum and Supervision of The School 

u 



of Education, University of Pittsburgh, came to the program with no 
curriculum experience. Diane Davis had been a substitute secondary 
school teacher. Her B.A. was in psychology from Youngstown State 
University, Youngstown, Ohio . 

Nicholas Laudato had completed undergraduate work at Carnegie- 
Mellon University in May, 1971. His B.S. was in mathematics. 

Unlike the other students, Sister Claire McCormick had a extensive 
background in education. She had been Dean of Studies and Assistant 

I 

Principal at St. Pius School, Atlanta, Georgia. Previously, she had 
been Principal of Notre Dame Academy, Villanova, Pennsylvania and a 
teacher of English and Latin. Her B.A. was in music from Trinity College, 
Washington, D.C., and her M.A. in education administration from Catholic 
University. 

Tommye Whiting had been Educational Coordinator for Job Opportunities 
for Youth in Houston, Texas, and previously a junior high school history 
teacher. Her B.A. was in French from Ofrerlin College, Oberlin, Ohio. 

Another student, Larry Hubka, did^not continue for the doctorate, 
but did receive a master's degree in curriculum and supervision, was a 
teacher and coordinator of distributive education. 

The students are probably the best indication of the effectiveness 
/of the materials. They are judged, by the professionals with whom they 
have worked at the University of Pittsburgh, and particularly at the 
Learning 'Research and Development Center, as thoroughly competent 
specialists in curriculum design. (See Appendix C, The Students, and 
Appendix D, Sample Student Work.) 

Diane Davis spent her internship with the External Studies Program 
at the University of Pittsburgh, working with University professors 

• . . , s 
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helping them to individualize th^Lr courses and make them self-instruc- 
tional so they could be offered extra-murally to students who could not 
Feadily come to the campus. Nicholas Laudato worked as an intern in the. 
Computer-Assisted Instruction Project and Sister Claire McCarmick .worked 
with the New Reading System (NRS Project), Both of these are projects 
of the Learning Research and Development Center at the University of 
Pittsburgh. Tommye Whiting- worked on an exploratory project* in social 
vstudies in the third grade at Prick School, a Pittsburgh public school 
which is a developmental school of the LRDC. 

Dissertation topics of the students are related ,to their internship 
experiences. A self --concept instrument for cultural-minority children 
grew out of Tommye Whiting 1 s internship in an inner city school, working 
on development of lessons in the concept of social interaction. Sister 
Claire McCormick is analyzing the NRS Reading Program of the LRDC to 
identify the instructional strategies it uses and to formulate principles 
and generalizations for the teaching of reading. Nicholas Laudato is - 
developing a program for teaching children to solve word problems by 
computer without computation. Diane Davis is developing a'mo<fel for 
evaluation and is evaluating some of the curricula Wing individualized 
and externalized, for the University of Pittsburgh's External Studies 
Program. 

In addition to their curriculum course work, which is represented 
by the twelve units which accompany this report, their other courses 
at the University, their internships and their doctoral research, each 
of \ these students has used his training to assist educators in the field. 
Diane Davis conducted some research in the schools with Dr. James Holland 
University of Pittsburgh, Associate Professor of Psychology and Research 




Associate of the LRDC. Nicholas Laudato larked out a behavior jnod if i-* • 
cation program for a Hebrew School, director who had discipline problems. 

>#Tommye Whiting worked as a consultant on individualization at the 

i • 

schools in Freedom, Pennsylvania. She consulted with the Braddock, 
Pennsylvania School District teachers who were attempting to quiet racial 
unrest stemming from recent integration -in the district. She worked., ^ 
also, with social workers for the Pittsburgh Board of Education, 

All four of the students assisted in conducting a day-long workshop 
for the Babcock (Pa,) School District. The R&D Project Director 
observed their interaction with the teachers and administrators and 
evaluated it. Clearly, each of these students can use his training 
effectively himself and can, also, teach others. 
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PROGRAM DEVELOPMENT: - PHASE 2 * 

Field Testing 

- Phase two began with a second test of the materials (first field > 
test), conducted with twenty-t;hree students, during the Fall and Winter 
trimesters of 1972. - 73. The course was offered by the University's 
Department of Curriculum and Supervision, through the External Studies 
Program of the School of General Studies. The students, post-baccalaureate 
adults who could not readily attend on-campus courses, came to the" campus 

tr 

only three times: for pretests, postests and interaction sessions. This 
externalized feature made it necessary to strengthen the course's self- 

V, 

> t 

instructional characteristics. 

f 

The course materials proved effective and suitable to tfie very 
diverse needs of these students. All but two students successfully 
completed the first trimester. Fourteen of these students enrolled in the 
second trimester of the course ,and completed it with eleven A and th^ee B 
grades. (See Summative Evaluation, p. 17 for more, evaluation data.) 
Flexibility in rate of progress was provided by use of a G (incomplete) 
grade for students who needed mote time to master the objectives. These 
G grades are removed when the student's curriculum projects have been 
completed and evaluated. (A description of th^ projects 6f field test 

i 

students is in Appendix E. ) 

The third test of the materials (second field test), began during 
the Winter, Term of 1973. It was conducted by. an instructor who had not 
been involved in the R&D Training Program, with sixteen students, who 
Qompleted the first trimester satisfactorily. Only three of this group 
of students continued the course for a second trimester $ in the Fall of 
1973, and all earned A grades in the course. 
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Sixty students took part in the fourth and final test of the materials 
from September to December, 1973. Most of them have successfully completed 
four un^ts. Many of thenr have indicated their ^intention to continue through 
,the entire instructional sequence, now Consisting of twelve units. 

Two new units were added to the External Studies course before the 
final field r test, replacing materials used in the pilot program on the 
background to instructional design. The University's Department of 
Curriculum and Supervision offered the highly flexible option of taking 
all twelve units externally, for nine graduate credits or any four of the 
units for three credits. * f r 

In addition to testing the two new units, the final ffeid test also 

c 

was an opportunity to try out the option ot^choosing freely those units 
needed by the student. This was founjl feasible and useful foi; some students. 
In addition, the appropriate posttest items for each student's unique 
combination of units were quickly located by the coding system used to 
identify units, objectives, levels and test items. They were then admin- 
istered by the University Testing Center to those stud ent^wfio could not 
conveniently come to the interaction session. 

The total number of students who took part in the pilot and field 
tests of these materials was one hundred and three, range of students 

in the field tests, their educational background and their needs, has been 
extremely broad. One student did not have a college degree. She had been 
trained as a practical nurse in a program for minorities. A few students 
had no previous graduate credits, but most students had graduate credits 
ranging from a few to several credits beyond the Master's degree, and 
there have been four post-doctoral students enrolled. The majority of 
the students have been teachers. There have been, also, guidance 

1G 



counsellors, reading specialists, community college instructors or 

assistant professors, registered nurses, a director of a nursing school, 

* -. 

and industrial training directors/ Several graduate students in 

Cufrriiulum and Supervision and Educational Communications also have taken 

• *»•* 

the course; '* 
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FORMATIVE EVALUATION 

Formative evaluation of these materials was accomplished by use pf unit 
rating sheets', which provided data for revision, and posttest's tcft each unit, ^ 
which gave evidence of the degree to which the materials successfully taught 
" the concepts and skills necessary to accomplish Unit objectives. 

The most extensive alteration in -the course materials, occurred during 

\ 

the pilot test when the course content and format underwent 4i major changes. 
However, the two revisions of the materials as a result of the field tests 
were largely the result of feedback from the students, (See Appendix B, 
this volume, for applicable data.) In addition to unit rating sheets and unil 
pdsttests, students" were given mailers on which they could list a question 
and receive a response from the instructor. Few of these were used but the 
questions, that were returned also suggested portions of the materials which 
required fuller explanations. 

'Not all students in the External Studies" field tests returned the unit 
rating^sheets and the responses of those who did were not consistent. Some 
students had problems understanding the articles which other students rated 
most valuable. Yet, it was possible to identify and correct weaknesses in 
the materials Qr to make use of specific suggestions for improvement. The 
External Studies^ staff,, compiled data for each of the ten units on subject 
matter relevancy, clarity of objectives, sufficiency of materials, interest 
of readings, effectiveness of self-scored tests, exercises and study guides. 
(See Appendix B, this volume, for this data.) 

It was interesting to note the difference in unit ratings under differ- 
ent instructors. The cbmments revealed an even greater difference in student 
opinion of the instructional materials then did the numerical ratings. Also, 
^18 of 23 students of instructor ofie* continued into the second trimester 



while only 3 students out of 16 continued from the class of instructor two. 

♦ 

Since the oourse is self ^^instructional and is meant to be readily 
disseminable it was important to find out why this difference occurred. With 
only three interaction sessions a trimester^ two of which w£re given over 
largely to testing, the student/instructor interaction could not have caused 
this difference. 

Analysis of the student posttests revealed a rather dramatic difference 
in amount and type of feedback. Instructor number one, whose students 
reported an overwhelmingly positive reaction to the units wrote extensive 
comments on every posttest. Instructor number fwo wrote relatively few com- 
ments. 

HJiere was also a clear difference in the type of feedback. Instructor 
two usually gave feedback in the form of questions or in a negative form. For 
examp|k,, "This is not backward chaining", buc without explaining what it yas 
or giving an example of the principle the student was attempting. Sometimes 
there was a simple comment, "Use tan mean apply, 11 but no explanation to clear 
up the student's confusion about how apply differs from analyze or synthesize. 
One student used both the terms, deductive and inquiry, in describing a lesson. 
The comment made by the instructor was, "Are these compatible?" 

Instructor number one, on the other hand, responded to a hierarchy that 
was not charted correctly by doing one or more possible alternative hierarchies 

and suggesting the student consider them. In response to an objective 

J, 

requiring the student. to "demonstrate awareness of the instructional strategies 
on which the program is based," the question was not only "By doing what?" 
An example was given, as well, "by defining the rationale for each." 

An objective for in-service training was stated as follows: "Teachers, 
upon completion of the in-service program, are to perform in the role of 
facilitator in,, the new environment." Teacher number one responded. "This is 




teacher will do. Example: in the role of facilitator as shown by contin- 
uous roving, reinforcing of process behaviors, suggesting how students can take 
responsibility for finding data, encouraging peer tutoring, etc. M 

An objective terminating a sequence on government functions stated that, 
the student should be able to "apply these ideas specifically to the U.S.' 1 
The student's objective was qualified by the instructor with the addition of 
"by describing how the government is organized and operates to perform these 
functions." Frequently Instructor number one suggested readings in the 



student's own field or specified the review necessary. She also commented 
favorably on every point it was possible to praise. 

As a result of this experience in field testing the materials, it was 
apparent that the instructor's manual must stress feedback and must emphasize 
the confusions which can result for students attempting self-instruction for 
the first time. 

Self-instructional materials have great possibilities for freeing 
professors to do research or to work with students on a one-to-one basis 



while the rest of the class is working independently. They offer an oppor-* 
tunity for people who live at a distance from a university to continue their 
education without lengthy commuting. A single course can be made flexible 
and adaptive enough to meet the needs^ of a broad range of students, to 
whose differences a professor lecturing in a classroom would not be able to 
adapt the instruction. However, the amount of feedback which might be 
acceptable when students and instructor are meeting each week is totally 
insufficient when the student cannot readily ask questions and receive 



The feedback should be as positive as possible. When something is 
clearly wrong, it must be labelled wrong, of course, but the correct response 



answers . 
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must he. Klvm. If, as in this course, there are many neither right nor wrong 
responses, an improvement in quality can be encouraged by suggesting several 
alternative responses: "Why not try this?' 1 "Would this be better, perhaps?' 1 
Whenever possible, each posttest should result in an individualized prescrip- 
tion for further reading, review, rewriting. 

The unit rating sheets, including student comments on what they liked 
^rest and least, were extremely helpful In eliminating unsuccessful articles. 
Some articles students found to be difficult were retained because they were 
the best available for a particular purpose. Often, students reread them 
later and changed their minds about their value. Some students failed to 
realize, in spite of being told, that they w£re to select the readings they 
needed to master the objectives. The Manual alerts the instructor to this 
student tendency. It was at the suggestion of several students, also, that 
a. glossary was added to the course materials. 

The students evaluated the usefulness of the various sections of the 
course format, but this data was only obtained from^the group of students 
who took both trimesters of the course in the Fall and Winter of 1972-73, 
They found the Overviews the most useful portion of the units. The terminal 
objectives were rated next, then the hierarchy. Prerequisite and study guide 
objectives were third in value, Posttests* rationale, and tasks and sources 
were fourth with the same number of students finding them "very useful," 
Those students who did not rate all of the sections mentioned here as "very 
useful" rated them "useful", except for one or two students who cated each 



ierul rated them userui » except tor one 
le "sfimewhat useful". The element of unit 



one "somewhat useful". The. element of unit format which/ was adjudged of 
least value (3 very useful, 1 useful, 5 somewhat useful and 3 not too use 
ful) was the bibliography. However, for students who needed additional help 
it was mentioned as indispensable. 

Use of the pretests to diagnose student needs was helpful for students 
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who intended to study the entire sequence as well as those who wanted to, work 
in only a few specified units. Particularly for adult students who have not 
been in school for awhile, but for all students studying self-instructional 
materials, it is not helpful to receive a pretest score. It was found that 
listing of the possible number correct and the number missed in each of the 
levels. also was not useful. The pretest was used primarily for self-placement 
and fdr a student not familiar with diagnostic testing it was painful to learn 
that most of the questions had not been answered correctly, even though that^ 
was the expected outcome. 

The procedure .finally used was a note to each student indicating the 
areas in which he seemed to be knowledgeable*sand those in which he should con- 
centrate his efforts. 

It was found that students come to the program strongest in terminology, 
the reform/movement, instructional theory and behavioral objectives. They 
are weakest in hierarchies, taxonomies, discipline elements, learning theory 
and criterion referenced testing. 

For the field test of the second half of the course the posttest for the 
first half was used to diagnose student deficiencies (if any) and to write a 
letter containing an individualized prescription for making up the deficiencies 
before continuing with the next unit. Each error in the posttest was recog- 
nized, not by indicating the student had made an incorrect response, but by a 
list headed "Notes on Posttest. 11 Each note was a statement of a concept, 
principle, generalization or method corresponding, correctly^fo the students 



erroneous response. The individualized prescription might be to reread an 

to 

overview, practice a behavior demanded in a unit posttest, review specified 
articles or read additional suggested materials in the student f s own field. 

Feedback directly from the students in the form of unit rating sheets 
and indirectly fronTtheir pretests and unit posttests were used for formative \ 
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evaluation of the curriculum program. It resulted in the addition of 
two units oin the background to instructional design because several 
students expressed a need for such an introduction to the course. It 
resulted in an Instructor's Manual to suggest supplementary activities and 
to alert the instructor to t'hose directions to which students often fail 0 
to attend. The Manual also explains some of the instructional strategies 
used in the design of» the materials'* 

Formative evaluation of the curriculum materials led to elimination 
of some articles, substitution of others, changing of some tasks or rewording 
of some postt^sts. It led to placing answer keys at the end of the unit 
rather than after each study ^uide, to extensive use of feedback, and to 
the addition of two units and an Instructor's Manual. 




SUMMATIVE EVALUATION 
The point made by Lindvall and Cox (1970, p. 58), in their monograph 
on the IPI program evaluation, applies equally to this program which is also 
based on a structured-curriculum model: "With this approach to program 
development there i& really no summative evaluation. There is -only summative 
evaluation _for a given stage, describing what results are produced by the 
program at that stage. 11 The results of the program as of its terminal date, 
November 30, 1373 are described here. Each new group of students expands the 
potential impact of the program, as each past group of students has provided 
formative evaluation data le iding to program improvement, A product-oriented 
curriculum in whic,h mastery is measured by ability to produce comprehensive 
instructional systems can on .y be evaluated by the quality of the products 
produced and the growth of the students from neophytes in curriculum to 
designers and developers of effective curricular components. 

The pretests, as noted in the chapter on Formative Evaluation, were 
used for diagnosis. Nevertheless, it is possible to measure the growth 
between pre and posttest with the first group of field test students who 
continued through the ten units then included in the program.^ For this 
group of students, the pretest was scored and the range was 23 to 42, the 
median 41. The posttest range was 74 to 96 with a median of, 91. Not 
included were, the three lowest-score students (46,60,70), wha, !< ele&t^rd to 
take incomplete grades, continued to work with the materials, and 
completed them with two A r s and one B in the following trimester. 

Validity of posttests in criterion-referenced measurement depends on 
whether or not they test the objectives. The unit posttests in this 
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course are valid because they all demand the s>ame behavior that if 
demanded by the unit terminal objective. 

In a structured curriculum, the student- must master the objectives 
of one unit before he can proceed to the, next. When the student is working 
independently and, in some cases, checks even his own posttest, progression 

is determined by him. However, the instructor checks enough of the posttests 

/ 

to monitor student progress and to prescribe progression to a new unit or 
more work. in the present unit . 

Test evidence has shown that pilot and field test students could 
perform the following behaviors: . * ^ 

I. * Analyze the structured-curriculum model in terms of adaptability to 
the needs and goals of a fast -changing world. 

II. Given an educational problem, identify theoretical formulations 
which couW,be used to select appropriate instructional strategies or to 
design instructional systems. 

III. Given a content scope and sequence for a one Semester course, 

; identify elements of the discipline structure. • . • 

III. Given elements of discipline structure and content scope and 
sequence, select content instances . \ ^ 

IV. Given a terminal objective construct a behavioral hierarchy and- 
code each component objective by the appropriate Bloom, "Krathwohl or Gagne % 
level. 

V. Given a student population and behavioral objectives, or self- 
selected population and objectives, specify appropriate instructional methods, 
medicT, strategies and classroom environment. 

VI. Given evaluation problems,, be able to describe how the curriculum 
designer would solye them.- . 

*ObjeGtives are numbered by^uiit. ^ 



VII. Given hypothetical administrative problems, prescribe, on the basis of 
administrative theory, the most effective means of solving them. 

VII. Given an innovative curriculum, plan an in-service training program 
to teach teachers to implement it. 

VIII. Draw up and specify in writing his own tentative, step-bystep pro- 
cedures for the design of instruction, which may be specific to his own area 
of instruction and which will be used for his own. design project. 

XI. Do curriculum content, concept and cQmponent analyses and chart and 
code his behavioral hierarchy by Bloom and Krathwohl or Gagne's levels. f 

X. Given a curriculum hierarchy (or a student-generated hierarchy) 
select the optimal testing points and write appropriate tests. 
- XI. Given (or self-selected) lessofr objectives CET tesfc and student 

characteristics; the ^student should be able to iden£ify an appropriate 
method, strategies, classroom environment and alternative media, fill out*a 
lesson writer's rationale and write a lesson. 

XII. Given a description of a management problem, solve it. 
XII. Given an individualized structured curriculum 6f his own design, 
design a complete management system. ^ 
» In addition to what the students of the program are able to do, the program 

¥ W - may "be further evaluated in terms of the five products which were promised in , 
>, / • the Dpc^mber, 1972 Report of the Project (p. 5> 
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,1. Pj&ts^nriel with speciaxi-z^ed training in curriculum design* 

> . . * ' 

The listing of ..what the students trained in the program can \* 
ftV do specifically on the preceding pages indicates that ,this promise 

ha^, been fulfilled. In addition, Appendix D exhibits work of the 
* pilot? group x>f students. "This k includes q pappr on- the External 
Studies -Dellttery System by Diane Davis; a paipefc* tm Computer Assisted 
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in Problem Solving: The Word Problem by Nicholas 

v 
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Laudato; an Analysis of NRS, the LRDC Redding Program by Sister 
Claire McCormick; and a series of cartoons to teach social interaction 
skills by Tommye Whiting. These four curriculum students evaluate 
their training in Appendix C. 

2. A tested training program and guides for conducting instruction in 
the program. 

The Curriculum Design Project has developed twelve self- 
instructional units, with pre and post-tests and an Instructor's 
Manual. This comprises a comprehensive course in the design of 
structured, adaptive, individualized curricula, and the manage- 
ment systems, administration and in-service training to implement 
them. It has been tested as described above. The four volume 
course accompanies this -Report. 

3. An operational model for a consortium of training agencies. 

In the Curriculum Design Project most of , the cooperation was 
between the Learning Research and Development Center and the f 
University of Pittsburgh School of Educatioi^, particularly the 

1 * 

Department of Curriculum and Supervision, .^his program would not 
have been possible without the cooperation and the willingness to 
try innovative approaches to training shown by the Curriculum and 
Supervision Department. This cooperation resulted in highly 
flexible, highly individualized courses of Study for the graduate 
students enrolled in the program. 

4. Tested approaches for evaluating training programs both on a program 
development basis as well as a student basis. 

The Unit Rating Sheets have proven an effective way to evaluate 
curriculum training materials (Appenddx- B) . Use of the structured 
mo<fel, behavioral hierarchies and criterion-referenced tests makes 
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possible straight-forward evaluation of student products to measure 
attainment of program goals. The terminal objective of each unit 
hierarchy can be posttested. 

Tasks, which include questions to direct student attention and 
aid retention, serve as curriculum embedded test items to help the 
^tudent monitor his own progress. These have proven very effective. 
The final outcome of * the operational test (the 1972 report suggested) 
may be contributions to theory and research on the development of 
training programs. * 

The curriculum project has hinted at several interesting 
principles of individualized, self -instructional curriculum design 
which should be further investigated by other researchers: 

1. It has lorijg been argued, particularly in response 
to critics of too much testing, that testing and teaching 
begin to merge in IPI and other structured curricula. The 

, format developed for this program virtually merges curricu- 
lum embedded tests with teaching sequences in the tasks 
which are the instruction. The optimal , arrangement of 
input (reading) and practice (answering questions or per- 
forming tasks) for adult learners should be investigated. 

2. The field tests seem to indicate that for a self- 
instructional course addressed to adultfe and curriculum 

/ I 
product-oriented, positive and specific feedback, with 

examples, is essential. For students who are highly 

motivated and self-directive it is possible to use the 

course without feedback from an instructor. The princi- 4f 

pie seems to be that self instructional training for 

development of curriculum products demands extensive 
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feedback to be optimally effective. 

V 

3. Pretests, used dlagnostlcally to Indicate which 
units' can be selected if the student needs only some but 
not all twelve of the units, secpn to^ provide the student 
with the greatest amount of direction if they are reported 
in terms of what the student knqws and what he needs to 
know. Reporting of scores is not helpful. Reporting of 
number correct out of the possible number is not helpful. 
Bbth discourage the student learning independently. 

4. For self -instructional materials, it is a very 
useful procedure to divide (and code) tests into units 
and, within units into taxonomic levels. This makes it 

1 possible to report in a meaningful way to the student in 
terms, of his weaknesses and strengths. It is helpful to 
a student who is managing his own instruction to tell 
him he needs to work on methodology or on application of 

, u ' 7 

curriculum design principles to a given problem or, per- 
haps, evaluation of curricular ^components in terms of 
external criteria. 
The Curriculum Design Program is unique in its comprehensiveness. It 
delves into all phases of ah individualized instructional system. It is 
readily transportable. Portions of it have been used at Governor's State 
University by a former colleague of the project director and by department 
members in the Department of Curriculum and Supervision at the University p 
Pittsburgh. 

Student attitudes are overwhelmingly positive. Laudato (Appendix C) 
praises its efficiency, "After attaining a more global picture, beginning 
again in detail was most instructive. In retrospect, the manner in which 



knowledge and skills Were slowly built upon one another was remarkably « 
efficient. ,, 

Product impact can be judged from the quality of the curricula being 
produced by the students (see Appendix E), by the impact on the students, 

w 

several of whom are continuing their graduate studies because of the 
interest aroused by the course; and by the number of students to whom other • 
departments at this university (and other universities) are recommending 
the course. From a first field test of twenty-three students in the Fall 
of 1972, the number enrolled in the Fall of 1973 rose to 60u In addition, 
two students of the'course, one from the pilot group and one from .the 
field test group, are working with university professors for the External 
Studies Program helping them to individualize and externalize their courses 
following the course design model. This is providing students who could not 
otherwise get to the University with carefully structured courses of quality 
least equal to those offered on campus. 
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THE TRAINING PRODUCT • 

The curriculum design and development course and instructor's manual 
described in this report consists of twelve self-instructional units divided 
into three malor concentrations. (Course outlines, Appendix A) The first 
section, which was left out of the first field test but was found to be 
necessary for many students, is on the Background to Instructional Design. 
Under the Goals of Education are study guides on Goal Setting; The Reform 
Movement and Individualization of Instruction. This examines forces influ- 
encing educational goals and provides historical perspective on instructional 
Innovations of the past two decades. ^ 

The second unit is on the Psychological Bases of Instruction which 
includes study guides on Learning Theories, Instructional Theories and 
Behavior Management. Here the major theories about how people learn and 
how we can help them to learn are described and compared, not to convert 
this very specialized course into a survey of the psychology of learning and 
instruction, but to establish the framework upon-which the Theoretical 
Rationale for Instructional Design, the succeeding section, is built. % 

The Theoretical Rationale for Instructional Design helps the student to 

understand why a structured curriculum design is based on the conceptual 

>• 

structure of a discipline; the purpose of stating objective^ in terms of 

\ 

observable behavior; and the need for explicit selection and recording of the 
instructional strategies used to move the student froni his entering to the 
desired terminal behavior. The kinds of evaluation used in designing and 
developing individualized instruction and the rationale for them follows. 
The concluding unit describes the differences in administration required for 
an adaptive instructional environment and the kinds of in-service trainings, 
needed for its staff. The procedures for designing and developing each of 
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these components are described and supported in theory in the study guides: 
Structure of the Discipline, Content and XN ^ice^ Analysis, Behavioral Objec- 
tives, Taxonomies, Component Analysis, Adapting Instruction to Learner Charac- 
* teristics, Instructional Methods, Medi'a and Strategies, Classroom Environment, 
Formative and Summative Evaluation in Curriculum Design, Administration *nd 
In-Service Training, 

The final section of the curriculum course is Applied Instructional De- 
sign. Beginning with Design Procedures and ending with Management System 
Design, this portion of the course requires that the student design- components 
of a curriculum on the structured curriculum model for individualization of 
instruction. The units between the design unit and the management system unit 
are Specification of Objectives and Structuring of Hierarchies,. Criterion- 
Referenced Test Construction and Lesson Writing. 

The student plans his own design procedures with the option of altering 
them as he proceeds, if they prove to be ineffective or inefficient. The 
study guides in this unit are Application pf Instructional Theory to Instruc- 
tional Design and A Process Mqdel for Instructional Design, which is the 
design model on which the course is written. (See Appendix A ) 

Other study guides in ^is section are Identifying and Writing Objectives, 
Structuring and Charting Hierarchies, Writing Test Items, Sampling the 
Objective Domain and Assembling Tests, Selection of Instructional Methods, ^ 
Media and Strategies, Constructing a^Lesson Rationale and Writing a Lesson,^ 
Feedback System and Staff Planning, procedures for eadh of these curriculum 
design steps are detailed and are pr^Sced by the students. Instructional 
strategies are not prescribed. The dl^sions which need to be made in 
designing instruction adaptive to individual needs, and the points at which 
they must be made, are clearly ^cplained. The presently available options, 1 

within the limits of the knowledge we x have about how we can help people leaifn, 

1 
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are set forth for the student to examine. ^ 

Each unit in the curriculum course begins with a rationale which 
establishes the relationship of the unit objectives to the overall^course 
objectives. The terminal objectives follow, stated in behavioral t;erms, and 
indicate the explicit behaviors which each student is expected to be able to 
exhibit at the completion of the unit. These terminal objectives are followed 
by prerequisite objectives to the unit. 

charted hierarchy of all the objectives of the unit gives the student 
a graphic representation of the structural relationship between the objectives 
and makes it possible for him to find out, at a glance, how far he has pro- 

r 

gressed and what remains to be achieved. 

The lesson materials are in the form of study guides, two or three to a 
unit. Each study guide represents a branch of the instructional sequence and 
begins with objectives. These lesson objectives are further defined in terms 
of tasks that are similar to curriculum embedded items for the student to 
perform. They help the student tio evaluate his own progress. They also are 
designed to assure the student's attention and direct it to the important 
information he should retain. Each task is listed parallel to a source which 1 
may be an article from a journal incorporated in the unit, a chapter in one 
of the four prescribed texts, or a portion of the unit overview. The sources 
are coded to the objectives they help the student to master. At the end of 
the unit, an answer key is provided for each study guide so the student can 
receive immediate feedback on his responses. The tasks may require the 
application of skills previously learned in the course to given design problems 
the analysis of elements of given curricula, or synthesis in designing curricu- 
lum components* 

The unit overview follows the study guides, giving an abbreviated version 
of unit content and usually including explanations and descriptions of concepts 
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and procedures not readily or briefly available in books or articles. 

Each unit has a bibliography with items also coded to the objectives 
to make it possible for the student to locate additional references to 
assist him in mastering any given unit objective. 

Finally, at the end of each unit is a posttest which may include an 
answer key, criteria or examples of appropriate responses that would indicate 
mastery of unit objectives. This unit test not only provides information 
to instructor and student on his progress and governs his decision to go o^ 
to a new unit, but also provides evidence of the effectiveness of the 
materials . 

This course differs from most curriculum training programs in that it is 
based on a process model for curriculum design (PIC) which is concept-centered 
and which includes procedures for a separate concept analysis, as well as 
content and component analysis steps in design procedures. Also, the model 
calls for application of research-based instructional strategies to instruct * 
tional design and record-keeping to monitor their effectiveness. Finally, 
the model demands systematic sampling of higher level skills of analysis and 
synthesis in each unit of instruction. 

Most of the procedures taught in the course are derived from experiences 
of research and development centers and the new field of instructional 
psycho logy. 

\ 

While this is based on a specific procedural model for instructional 
design, it is in n6 way limited in the variety of theories from which the 
designer can borrow, or the creative approaches to instruction which it is 
possible to employ in building on the model^Btructure . 

The entire course has been revised on the basis of the feedback from 
pilot and field testing. An Instructor's Manual has been written to identify 
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some of the misunderstandings which could arise when students are studying 

independently and to suggest how the instructor can prevent problems by his dwn 

directions and the kind of feedback he provides on the student's work. The 

V 

Manual also suggests supplementary activities. 

There is d pretest for the entire course which is coded according to 
skill levels and units so that it is possible to pretest on any unit or group 
of units or the entire course. It is also possible to pretest on particular 
skills: for example, curriculum analysis. 

This product is twelve highly adaptive, highly^ legible, self-instructional 
units in curriculum design and development, a course pretest, unit posttests 
and afi instructor's manual. 

Potential users of the course ar# R&D Centers for staff training, In- 
Service Teacher Training Institutes, University Undergraduate Schools of 
Education, University Extra-Mural Post-Baccalaureate Programs (See Appendix E ■ 
fpr the broad range of students this course has served), Graduate Education 
Programs for Curriculum Specialist, for Master's Degree or for Doctorate in 
Curriculum. 

A Publisher's Alert and a Request for Proposals have been sent to The 
National Institute for Education. 
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Appendix A 

PIC: A Process Model for 

\ 

the Individualization of Curricula 
(The Design Model) 
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PIC: A PROCESS MODEL FOR INDIVIDUALIZATION OF CURRICULA 

Doris T. Gow 

A. 

University of Pittsburgh 

INTRODUCTION 

The Carnegie Commission on Higher Education (1972) has predicted 
that "off-campus instruction of adults may become both the most rapidly 
Expanding and the most rapidly changing segment of post secondary 
education (p. 4)." Because there are few self-instructional materials 
capable of meeting this need, they have urged that learning technology 
centers be established to engage in design, planning and production 
instructional units for use by participating institutions and extr 
educational systems (pp. 55-56). 

This paper describes a model used to develop and package curriculum 
design training materials and University of Pittsburgh External Studies 
courses. It suggests that the model wiAiid be effective and Efficient 
for the production of the instructional units proposed by the Carnegie 
Commission. The training materials themselves would be appropriate to 
train personnel for the learning technology centers the Commission has i 
advocated . 

Called PIC, or the Process Model for the Individualization of 
Curricula, the model combines structured curriculum design components 
with procedures for developing curricula which emphasize the structure of , 

\ 

the discipline. These procedures focus on process by sampling in each 
uni^t all skill levels in order to promote the building of independent 

i 
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learning capabilities; systematic application of research-based instructional 
strategies to the design of instruction; and use of independent inquiry 
within a structured model for instruction. 

( 

EVOLUTION OF THE MODEL 

This curriculum model was developed at- the Learning Research and 
Development Center at the University of Pittsburgh to provide curriculum 
design training materials and to train curriculum design specialists. 
The training was federally funded to help alleviate the well-documented 
shortage of trained personnel for R and D Centers (Chase, 1964; Clark 
and Hopkins, 1969; Gideonse, 1970). 

Appropriately, since the LRDC had developed Individually Prescribed 
Instruction (IPI) and the Primary Education Project (PEP), both structured 
curti^uia, the design model selected for the training program was based 
on the structured or adaptive curriculum model. Lindvall and Cox (1969) 
have identified the components of this model as: 

1. Sequences of instructional objectives to define the 
curriculum 

2. Instructional materials to teach each objective 

3. An evaluation procedure for placing each pupil 



at the 

4. A plan 
study 

5 . A proce 
progress 



appropriate point in the curriculum 
for developing individualized programs of 



lure for evaluating and monitoring individual 



Procedures developed for application 'of the structured curriculum 
model to social studies (Gow, 19 72) were selected for the design of 

the curriculum training materials, with a self-instructional format 

\ 
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(Gow, 1972/73) which would allow students to assess their own needs and^ 
' develop their own programs of study. 

When the University of Pittsburgh inaugurated a pilot external 
-studies project, a major part of these self-instructional materials bfecame 
a post baccalaureate course in the design and development of ^utricular 
materials and the model for individualization of other University courses 
for extra mural, self-instructional studies. The PIC Model's content 
and component analysis procedures based on structure of the disciplines 
seemed to be uniquely suited to the complex subject matter of higher 
education, and the format was especially appropriate to meet the problems 
of extra mural education. 

THE MODEL fl> 

The model, which evolved out of th|e efforts described here, is 
.termed a Process Model for Individualization of Curricula to distinguish 
it from ^ther structured curriculum models. While it is comprised of 
the usual, gtructured-curriculum components and procedures (Gagne, 1965; 
Mager, 1962; Nitko, 1972; Bloom ^ 1956; Resnick and Wang, 1969) there are 
fundamental differences in the techniques for establishing instructional 
sequences and structuring hierarchies. In addition, .there is a 
systematic attempt to apply the information tfe now have from the expanding 

« 

field of instructional psychology to the design of curricula. The model % 

• ■ S 

itself is adaptive and provides a vehfcle for the^ incorporation of new 
knowledge about the learning process as it is acquired. 
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Thje persuasive case for the teaching of the^ fundamental concepts 

and inquiry, to promote transfer and acquisition of intellectual skills, 

was made as far ba<ip§s 1929 by Whitehead (p. 21) and 1938 by Dewey 

(p. 30), but it was Bruner 's The Process of Education (1960) which had 

a major impact on the curricula created during the Reform Movement of 

the Sixties. In Bruner 's words: 

"To understand something as a specific instance of a 
more general case ... is to have learned not only a 
specific thing, but also a model for understanding 
other: things lJJce it th^t one may encounter. 11 

(Bruner, 1960, p.2£) ' 

To know method, anojther element of structure, is to know how to 
find out more about a subject. The processes the student uses are the 
intellectual skills he needs if he is to' acquire, organize and use the 
information fundamental to the discipline. 

In addition, development of the concepts, which are elements' of 
a discipline, is a process itself. Concept accommodation takes place as 
concepts become integrated into the student's frame of reference. 
Concepts grow and change instead of remaining static over time. Also, 
concepts are idiosyncratic because each individual's exp$rienc&^As— unique . 
Individualized instruction based on concepts is therefore intrinsically 
individualized as well as adaptive to each student's needs. 

Many quality curricula have been built on structure of disciplines 
in the past decade. (AAAS Science; BSCS Biology; SMSG Math; Taba Social 
Studies; Senesch Economics). Although Bruner seems to have become 
somewhat disenchanted (1971). with the ideas espoused by The Process of 
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Education , at least for the elementary and secondary years, his attitude 
apparently reflects a new concern for the importance of initiative and 
motivation )to learn. 

At the University level, however, discipline structure remains 
an essential consideration in curriculum design. Structured university 
education should focus on the processes which provide the student with 
the ability to learn independently. The PIC Model uses the fundamental 
concepts, principles, generalizations and methods of the discipline as 
the foundation for curriculum design and the basis for development of 
objective hierarchies and sequencing of instruction. 

The Taxonomies of Bloom (1956) and Kxathwohl (1964) are also 

used in the development of hierarchies. When curricula are analyzed, it 
*» 

is found that they seldom teach and test beyond, the third level of 

-* • 
Bloom's Taxonomy (Cox and Wildemann, 1970, pp. 24,26,38,42). Yet, we 

know that higher level skills transfer or facilitate learning (Gagne, 

1971, p. 116; Bloom, et al., 1971, p. 122). The PIC Model, therefore, 

uses the Taxonomies as tools in the design process. 

A further essential procedure to maximize the effectiveness of a 

structured-curriculum is the systematic application of research-based 

instructional strategies appropriate for the student, the subject matter 

f r 

and the objectives. The model borrows, pragmatically, from developmental, 
behaviorist or cognitive theory for selection of these strategies. 

The Process Model prescribes structured curriculum components 
and describes procedures for building each component. The curriculum 



designer is encouraged to establish his own procedural sequence, rather 
than follow a 'linear systems approareh, because on-the-job experience haar 
revealed that curriculum design is a highly complex and personal process. 
It involves backward steps to revise a previously writtili portion in the 
light of each forward step. „ 

i 

DESIGN PROCEDURES 

It is in identifying the first component of the structured 
curriculum, sequences of instructional object ives to define the curriculum, 
that the PIC Model differs appreciably from most design procedures. 

The PIC Model uses the usual content and component analysis > 
procedures (Gagne, 1968) but adds to them: (a) concept analysis and 
(b) systematic sampling of skills. 

Content is defined for this model as the peop^fe, informati^p, 
events and data at the knowledge level (1.32 affd below) of the cognitive 
Taxonomy . Content is selected because it represents the positive or 
negative instances of the basic concepts which are most appropriate for 
the student population and for the level of the instructional sequence 
based on our present knowledge of concept learning. A variety of concept 
instances may be Identified making it possible for the specific content 
he will study to be chosen according to the individual student's 
interest. Instead of concepts being developed as a by-product of the 
study *of Content, the facts, events and data of content are used to 

/ 4 

encourage conceptualization. The focal point is the concept. 
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For an already existing course which is ieing individualized, 
the course outline^rovJ^es the content scope and sequence. For a new\ 
course, it is necessary to do a content analysis, ordering the subject 
matter chronologically, by topic or by whatever logical organization the 
curriculum designer has selected. 

The fundamental concept^ of the discipline which the course 
teaches are identified. This process may be called a concept analysis 
since it identifies the concept hierarchy or inter-relationships of the 
conceptual structure. A concept analysis produces a hierarchy of sub- 
concepts, concepts, principles and generalizations for each unit of 

J 

instruction. 

Tb perform a concept analysis, the curriculum designer begins 
with a generalization and works backward asking what principles the 
students must know and b^able to apply in order to understand this 
generalization. Then he aslffs what concepts he must know, and be able 
to identify examples of, in order to understand this principle. In this 
fashion, he works back to subconcepts that the student may be expected 
to know and be able to use. Often, specification of these elements 
of the discipline structure leads to addition of concept exemplars or 
reordering of content. 

m 

The PIC Model requires explicit identification of skills and 
methodology of the discipline and the use of .Bloom's Taxonomy 'to select 
intellectual processes to incorporate into the course. The Taxonomy 
is used, in other words, to generate objectives. The process of 
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ssing the objectives of the course behaviorally involves expressly 
sampling higher taxonomic level skills. 

The curriculum designer combines the products of the content and 
concept analyses, merging them with the selected skills and expressing 
the behavior in objectives which state what the student does , under what 
conditions and how well he doe^ it (Mager, 1962). In constructing the 
instructional hierarchy, all of the following are considered: logical 
order of content and concepts, sequence of elements of discipline 
structure, and taxonomy levels of skills. To teach students the processes 
of learning, the intellectual skills and methods of the discipline are 
practiced on content and concepts. 

In*his early writings, Gagne used the terms "concepts 11 and 
"principles 11 in his hierarchies. Later, he changed concepts and principles 
to concept learning or classifying and applying rules or principles. 
The original use of these terms and the subsequent change reflect the 
problem which the PIC Model attempts to solve. 

Concept^* may be both knowledge and process. Therefore, they are 
inherently different from both content, as facts, events, people and data, 
and skills . When content includes concepts this dual function of .\ 
concepts . causes great difficulty in structuring hierarchies, especially 
when dealing with complex subject matter. By separating content analysis 

from concept^ analysis and component analysis this problem is eliminated. 

si % 
1 The Second component of a structured-curriculum, instructional 

materials to teach each objective , requires the curriculum designer to 
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usq his teaching skills to move' the student from his entering^ behavior 
to mastery of the terminal objectives of each lesson. It is in the 
degree of specificity of design strategies and the technique of recording 
the rationale for them, that the process model differs from most structured 
"curriculum models. 

The PIC Model requires that definition of each pedagogical 
decision of lesson writer be explicit. A planning form calls for listing 
of the objectives' prerequisite behaviors and the activities the student 
will participate in or tasks the student will perform. Most importantly, 

^ the lesson rationale defines and explains the method, mode, setting and 
instructional strategies used in the lesson. This requires a systematic 
examination of each element which must be matched to achieve maximally 
effective instruction. * 

The careful specification of strategies, and the systematic 
attempt to relate the lesson design to the requirements of the student 
and the subject matter and to justify that match by means of a lesson 

^✓rationale eases formative evaluation procedures. It becomes possible 
to locate and change instructional strategies found to be ineffective, 
without altering other elements of the curriculum. The lesson rationale 

4 

technique can facilitate evaluation of different strategies and develop- 
ment of more effective instructional materials. • 

The PIC Model does not differ appreciably from other such models 
in its procedures for development of the final three components of 
the structured curriculum: An evaluation procedure for placing each 
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pupil at the appropriate point in the curriculum; a plan for developing 
individualized programs of study and a procedure for evaluating and 
monitoring individual progress . It employs the usual criterion-referenced 
tests: diagnostic, placement, curriculum embedded or unit sub-tests and 
post-Oasts, 

The model does, however, include a procedure for selecting testing 
points to make testing more efficient, S/ charting each unit objective 
hierarchy (Nitko, Swanson, 1968), it is possible to select the optimal 
testing points which can reduce the number of necessary tests, (See 
Appendix III), Since it can be assumed that mastery of earlier sequential 
objectives has preceded mastery of later ones, branch terminus objectives 
may, in many cases, be optimal testing points. 

Sometimes there are two terminal objectives; one a cumulative or 
culminating objective and the other a synthesis objective, A cumulative 
objective is essentially the sum of all the other objectives, A synthesis 
objective goes beyond this to incorporate the student's own perspective, 
solution or organization of the knowledge and skills of the hierarchy. 
The decision on whether to test one or both or whether to demand mastery 
of both, depends on diagnosis of the student's present requirements. 
For example: Suppose a student were studying social studies and there 
were two sequential objectives at the top of a charted unit hierarchy. 
He mastered the first which was cumulative, but could not master the final 
objective which required him to formulate a generalization. More examples 
of the concept in additional optional materials might lead him to the 
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generalization, or it might be deferred until his next encounter with 
the concept. (See Appendix IV) 

Finally, the PIC model, because it is a process model, can be 
used to produce structured curricula written in advance of the instructional 
event and still feature open-ended individual inquiry as a possible 
desigp option. Since the focus is on processes, and instruction is care- 
fully sequenced and written to teach process, criteria can be established 
for unique responses resulting from independent research or creative 
production. The product can be evaluated in terms of the evidence it 
provides of the processes used to produce it. Self-evaluation can be 

accomplished by use of criteria and process check-lists. 

*■ 

THE FORMAT OF THE CURRICULUM COURSE 

Each unit of the curriculum course based on the PIC model includes 
objectives, a charted objective hierarchy, study guides, answer keys, an 
overview, pertinent reprints of journal articles, a bibliography and a 
post-test. Although the format is particularly suited to mature students 
capable of self-direction, it can be used for instruction at any grade 
level, including computer assisted or teacher or aide monitored instruction, 

A 

since all components of the instructional package are cross-coded. 
Curriculum-embedded test items, study guide tasks and sources for these 
tasks are cbded to the objectives. 

Students are guided in their choice of objectives by the pretest 

and their own aims. Individual differences among extra mural students 

t 

may be expected to be greater than among university undergraduates or 
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graduate students. Therefore, the pretest directs students to remedial 
units and permits them to "test out", of units they do not need. 
Additional sources, listed in' the bibliography, can be used to remedy 
deficiencies in comprehension or skill revealed by failure to master test 
items or study guide tasks. 

This format makes it possible for the student to determine his 
own placement in the program, plan his own program of study and monitor 
his own progr^^s. 

INDIVIDUALIZED APPLICATION OF THE MODEL 

The curriculum course which teaches the design model described 

r 

in this paper also teaches the instructional theory on which the PIC 
model is based. The procedures for designing curriculum components 
range from simple procedures to convert a traditional course to an 
individualized one to highly complicated procedures for creating an 
adaptive instructional environment. Knowledge of the theoretical basis 
for the design of instruction makes it possible for the student curriculum 
designer to make an informed judgment about the degree of structure he 
needs and wants to incorporate in his course and to selectively study as 
much as he needs to know to accomplish those ends. 

For example, the instructional designer who will be employed at 
an R and D Center would need to learn some of the highly sophisticated 
techniques for validating instructional hierarchies, while this might 
not be necessary or feasible for university professors who cannot devote 
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considerable time to individualization of their courses. Imposing these 
techniques on teachers and curriculum specialists in schools might 
discourage further efforts towards structuring effective individualized 
curricula. Rather, hierarchies can be considered tentative until empirically 
validated. An advantage to this latter stance is that it reinforces the 
attitude that curriculum development is a process and curricula are re- 
visionary rather than static. 

SUMMARY 

The predictions and recommendations of the Carnegie Commission on 
Higher Education of the trend toward off-campus instruction of adults and 
the need for de^ign of instructional units to meet the expected expansion 
suggest that a model is needed for the complex subject matter of the 
university. Such a model would contribute to the effectiveness and 
efficiency of higher education and particularly of extra mcral adult 
education. 

The Process Model for Individualizing Curricula (PIC) described 
in this paper, focuses on the structure of disciplines which make it 
appropriate for complex university-level content. Its trial run as a 
graduate course, developed by using the processes it teaches, has shown 
its effectiveness for teaching learning and instructional theory as well 
as applied curriculum design skills. This sterns to support its usefulness 
for both theoretical and applied courses. 

The highly structured unit design and self-instructional format 
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recommend the total design package for use in in-service teacher training 

and for master's and doctoral programs as well as for teaching extra 

i 

mural university curriculum design in any field. 
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APPENDIX I 

PROJECT TO DESIGN NEW PATTERNS FOR TRAINING R&D PERSONNEL IN EDUCATION: 
- CURRICULUM DESIGN AND DEVELOPMENT PROJECT 

Course Outline: The Design of Individualized Instructional Curricula 

I 1 

Background -to Instructional Design 

ij 

I. Goals!; of Education 

£ 
r 

A. j Goal-setting 



B. r ' The Reform Movement 

K 

C. I Individualization of Instruction 

II. Psychological Bases of Instruction 

A. Learning Theories v^, 

B. , Instructional Theories 
CJ Behavior Management 

Theoretical Rationale for Instructional Design 

III. ^he Subject Matter y 

i 

A. Structure of the Discipline 

B. ' Content Analysis 

IV. ( The Skills 

A. Behavioral Objectives 

B . Taxonomieis 

C. Compoja^nt Analysis 

V. The InstAiction 

A. Instructional Methods aid Strategies 

B. Media 

C. Classroom Environment 

VI. Evaluation 

A. Formative 

Feedback 
Field Testing 
Dis semina t ion 



B. Summative 
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VII. School Organization 

A, Administrative Theory and Practice 
.B. In-Service Teacher Training 
Applied Instructional Design 
VIII. The Design of Instruction 
A. Design Procedures 

,B. Specification and Structuring of Objectives 
C Criterion-Referenced Test Construction 

D. Selection of Instructional Methods, Media, Strategies 

and Setting 

E. Lesson Writing 

F. Management System Design 

Curriculum Synthesis 

IX. Instructional System Development Project 



. APPENDIX II 
External Studies Cpurse 

Curriculum Design and Development A Process Modfel for Individualization 

Background to Instructional Design 

I. Goals of Education 

Study Guide \. Goal-Setting 

Study Guide 2. The Reform Movement 

Study Guide 3. Individualization of Instruction 

II. Psychological Bases of Instruction , 
Study Guide 1. Learning Theories 

Study Guide 2. Instructitffiip. Theories 
Study Guide 3. Behavior Management 
Theoretical Rationale for Instructional Design 

III. The Subject Matter 

Study Guide 1. Structure of the Discipline 

* 

Study Guide 2. Content and Concept Analysis 

IV. The Skills 

7 

Study Guijde 1. Behavioral Objectiv^&^i—- 
Study Guide 2. Taxonomies, 
Study Guide 3. Component Analysis 

V. The Instruction 

<* 

Study Guide~±T~ Adapting Instruction to Learner Characteristics 

Study Guide 2. Instructional Methods, Media t and Strategies 

Study Guide 3. Classroom Environment 

VI. Evaluation * 

Study Guide 1. Formative and Summative Evaluation in Curriculum 
Design 



ERIC 



VII. School Administration and In-Service Training 
Study Guide 1. Administration 

Study Guide 2. In-Service Training 

Applied Instructional Design 

r 

f 

VIII. Design Procedures 

Study Guide 1. Application of Instructional Theory to Instructional 
Desiign 

Study Guide 2'. A Process Model for Instructional Design 

IX. Specification of Objectives and Structuring of Hierarchies 
Study Guide 1. Identifying and Writing Objectives 
Study Guide 2. Structuring and Charting 'Hierarchies 

X. Criterion-Referenced Test Construction 
Study Guide 1. Writing Tes£ Items 

Study Guide 2. Sampling Objective Domain and Assembling Tests 

XI. Lesson Writing 

Study Guide 1. Selection of Instructional Methods, Media, Strategies 
and Setting * 

I . ^ 

Study Guide 2. Constructing a Lesson Rationale and Writing a 
Lesson 

; 

XII. Management System Design 

Study Guide 1. Feedback System 
* Study Guide 2. Staff Planning 
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APPENDIX III 
Examples bf Charted Hierarchies 



Hierarchy 



ELEMENTS OK DISCIPDlNE ] } 
STRUCTURE AND CONTENT 
SCOPE AND SEQUENCE 



SELECT CONTENT INSTANCES 



CONTENT SCOPE 
FOR 

ONE SEMEST 


AND SEQUENCE 
A 

ER COURSE 


IDENTIFY ELE 
DISCIPLINE 


MENTS OF THE 
STRUCTURE 



10 



IDENTIFY ONE OF EACH 
ELEMENT OF STRUCTURE 
OF HIS OWN DISCIPLINE 
HE WOULD TEACH AT A 
SINGLE GRADE LEVEL 



CONTENT SCOPE AND 
■SEQUENCE TOR ONE 
SEMESTER COURSE 



SELECT METHOD AnO 
ORGANIZATION FOR 
COURSE 



PASSAGE FRO* 
STANDARD TEX 


1 ANY 
T BOOK 


IDENTIFY ELE 
OF STRUCTURE 
THE DISCIPLI 


MENTS 

OF 
NE 



PORTIONS 
NATIONAL 
PROJECTS 


OF 

CURRICULAR 


ANALYZE M 
IN TERMS C 
AND ORGAN I 


ID CLASSIFY 
)F METHOD 
ZATION 



COURSE 

DESCRIPTIONS 



DISCRIMINATE 
CONTENT-ORIENTED 
PROCESS, DISCOVERY 
GUIDED DISCOVERY, 

PROBLEM-SOLVING 
METHOD 



COURSE * 
DESCRIPTIONS 



DISCRIMINATE 
CHRONOLOGICAL, 
POST-HOLING, 
SURVEY, TOPICAL 
ORGANIZATIONS 



DEFINE AND GENERATE 
EXAMPLE OF EACH ' 
ELEMENT IN PROCESS 
CURRICULUM 



IDENTIFY EXAMPLES 
OF CURRICULAR 
METHODS AND 
ORGANIZATION 
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SUBJECT MATTER 
OF A TEXT-BOOK 



IDENTIFY A REASON- 
ABLE CONTENT SCOPE 
AND SEQUENCE 



D. Hie rarchy 



11 



Curriculum 
hie rarchy 


12 


Student generated 
hierarchy 


select optimal 
testing points 
and write 
appropriate tests 


select optimal 
testing points 
and write 
appropriate tests 













Given curriculum hierarchy. 
and selected testing points 



write an appropriate test 



Obje 


ctive 


writ 
vali 


e a 

d test 



Objective 


5 test 


establish 


mastery 


criterion 


that will 


maintain 


test 


validity 
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Objective & test 

evaluate testfs validity" 



state the conditions 
necessary for a valid 
criterion-referenced 
test 



Test items* 



student will Evaluate 
and rewrite faulty ones 




state purpose 
of testing in 
structured 
curriculum 



(Prerequisite, Unit II, Objectives) 



Curriculum 

Hierarchy 

select optimal" 
testing points 



Curriculum 
hierarchy and 

testing points 
evaluate selection 
of these points in 
terms of appro- 
priateness, feasi- 
bility , efficiency 



8 



1 



name and define 
purpose of each 
type of test in 
a structured 
.curriculum 



^Multiple choice, tme-false, short answer, matching, essay. 
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APPENDIX IV 

NOTE: The following example of optimal testing points shows the structure 
only, not the specific objectives, in order to emphasize the general 
principle. This is done because, in specific instances, the testing points 
are a matter, of judgment based on the. objectives themselves. However, the 
optimal testing principle, used with discrimination, can increase testing 
efficiency. 



Hierarchy and Testing Points 



. 11 



10 



3 



5 



For pretest 4, 6, 9, 

For CETs 4, 6, 9, 

For post-tests 10, 11 or just 11. 

For placement 1, 2, 4, 6, 7, 9. 
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Explanation 

The testing of 4, 6 and 9 for the pretest would narrow the options 
sufficiently to be an economical procedure. If the student failed 4 he 
would start in 1. If he mastered 4 and 6,- and failed 9 he would start 

J 

in 7. If he mastered 4> & and 9, he would be given the post- test. 

The CETs would be given for 4, 6 and 9 because they represent 3 
different sub-hierarchies. Giving 10 would not indicate which of these 
3 he l^d failed to-master." 

The post-test need only be on the final objective if it is a 
cumulative objective which demands behavior that is essentially the sum 
of all thfe'^ther objectives. However if 10 were cumulative and 11 went 
beyond to synthesis, it might be wise to test both 10 and 11 td ascertain 
whether the student had mastered^ everything to and including 10. If he 
had, but failed 11, he would probably profit from more practice with 
different materials rather than repetition of the *same lessons, or the 
final synthesis objective might be deferred. 
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Appendix B 
Unit Rating Sheets 

Pilot test of materials 
Field test of materials 
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UNIT CRITIQUE FORM 
(Pilot Test of Materials) 



Unit No. 



Each of the following categories represents a particular aspect or 
section of the instructional! unit. "Rationale," for example, refers to the 
rationale for the unit content presented at the beginning of each unit. Each 
category (section) of the unit should be evaluated on such bases as whether 
it was "clear, M "concise?," "effective in its goal/ 1 "weak," etc., i.e., on 
both positive and negative characteristics. Suggestions for revision should 
be included using specific examples. If any particular category is not 
relevant to the particular unit, omit comment. Be specific . 



Rationale 



Terminal objectives 



/ 



Prerequisite objectives 



Intermediate objectives 



Evaluation Procedures 



Pre-Post-Tests and CETS 



Hierarchy 
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Activities 



Speakers 



Bibliography 



Abstract /Exemplars 



Worksheets/ exercises 



Supplementary materials 



Class sessions 



Tutorials /individual meetings 
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UNIT RATING SHUT : C 5 S 850 

Design and Development of Curricular Materials 



(Field Test 
of Materials) 



To assist us in improving this instructional 
unit for use by future students, please take 
a few minutes at the end of the unit to 
complete tint- record sheet, and return it in 
the enclosed* envelope to the UhSP office. 
The individual forms will not be seen by the 
instructor. 



Name 

Date_ 

Unit 



T i ino 

Spont 

on Unit 



Please rate the following by circling the rating number that you 
describes each item: 




Very 
Hitch 



Subject matter relevancy to your own 
interests (professional or otherwise) 

Clarity of the unit objectives 

Degree to which the unit materials 
provided the information nccsssary 
to accomplish the unit objectives 

Interest value of the unit materials 

Difficulty level of the unit 



5 
5 



4 

4 
4 

4 
4 



A verage 
3 

3 
3 

3 
3 



?. Did you have problems with any of the unit objectives? Yes 
If yes, list the objective numbers here 



fee] 



Low 



2 
2 

2 
2 

No 



Very- 
Low 



What did you like best about this unit? 



4. 



What die! you like least about, this (including articles that were least 
productive)? 



What specific changes would you recommend for improving the effectiveness of 
this instructional unit? 
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Summary Unit Rating Sheets 
Unit I (now III) 
(Instructor 1) 



Very Very 
Good Good Average Poor Poor 



1. The subject matter relevancy 

to your own interests (pro- 4 
fessional or otherwise) 


5 


7 




• 


2. The clarity of objectives 4 


7 


5 






3. ine degree to wnicn the oojec— 
tives and materials provide 
the information necessary to 
accomplish the unit objec- 
tives 


6 


5 


1 




4. The interest value of the 
readings 


• 5 


5 


6 




5. The assistance of self- 
scored tests to self 5 
evaluation 


7 


5 






6. The instructional value of 

the exercises (if applicable 2 
to this unit 


4 


2 


1 




7. The instructional value of 
the study guide(s) 


10 


3 







r 

1 
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Summary Unit Rating Sheets 
Unit; II (now IV) 
(Instructor 1) 



Very 
Good 



Good 



Average 



Very 

Poor • Poor 



1. The subject matter relevancy 
to your own interests (pro- 
fessional or otherwise) 



2. The clarity of objectives 



3* The degree to which the objec- 
tives and materials provide 
the information necessary to 
accomplish the unit objec- 
tives / 



4. The interest value of £he 
readings / 



5. The assistance of sel, 
scored tests to self 
evaluation • / 



6. The instructional vplue of 
the exercises (if Applicable 
to this unit 



7. The instructional/ value of 
the study guide (s) 
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Summary Unit Rating Sheets 

t 

Unit III (now V) 
(Instructor 1) 



Very; ' * Very 

Good Good Average Poor Poor 



1'. The subject matter relevancy 

to your own interests (pro- 7 
fessional or otherwise) 


6' 




* 




2. The clarity of objectives 5 


7 


1 


• 




3. The degree to which the objec- 
tives and materials provide 
the information necessary to 4 
accomplish thfe unit objec- 
tives " ^ 


6 


2 






4. The interest value of the ^ 
readings 


2 


2 






5. The assistance of self- 

. scored tests to self ^ 5 
evaluation 


6 


2 


Y 

\ 




6. The instructional value of 

the exercises (if applicable 4 
to this unit 


7 


2 






7. The instructional value of 

the study guide (s) • 5 


7 


1 




> 
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Summary Unit Rating Sheets 



Unit IV (now VI) 
(Instructor 1) 



Very * Very 
Good ^ Good Average Poor Poor 



1. The subject' matter relevancy 

to your own interests (pro- 7 
fessional or otherwise) 


3 


y 






* . 

2. The clarity of objectives g 


4 








3. The degree to which the objec- 
tives and materials provide 
the information necessary to g 
accomplish the unit objec- 
tives 


»> 


1 . 




T 


— \ 

A. The interest value of the 

readings 3 


5 


2 






5. The assistance of self- 
scored tests to self * 4 
evaluation 




1 







6. The instructional value of 

the exercises (if applicable 3 
to this unit 


3 








7. The instructional value of 

the study guide (s) 5 


5 
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Summary Unit Rating Sheets 
Unit V (now VII) 
(Instructor 1) 



Very Very 
Good Good Average Poor Poor 

• S^r 



The subject matter relevancy 
to your own interests (pro- 
fessional or otherwise) 



2. 

3] 



The clarity of objectives 



The degree to which the objec- 
tives and materials provide 
the information necessary to 
accomplish the unit objec- 
tives 



The interest value of the 
readings 



The assistance of self- 
scored tests to -self 
evaluation 



The instructional value of 
the exercises (if applicable 
to this unit 



The instructional value of 
the study guide (s) 



Summary Unit Rating Sheet 
Unit VI (now VIII) 
(Instructor 1) 



Very Very 
Good Good Average Poor„ Poor 



1. The subject matter relevancy 
to your own interests (pro- 
fessional or otherwise) 1 


3 




+ 1 




2. The clarity of objectives y 

< 


1 


1 


1 




3# The degree to which the objec— 
tives and materials provide 
the information necessary to 3 
accomplish the unit objec- 
tives 




I 


1 


— t 


4. The interest value of the 

readings ^ ^ 


2 




i 




5. The assistance of self- 
scored tests to self 3 
evaluation 


1 




i 




6. The instructional value of 

the exercises (if applicable * 
to this unit 


1 


1 






7. t The instructional value of 

the study guide(s) 3 


2 


1 







/ 
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Summary Unit Rating Sheet 
Unit VII (now IX) 
(Instructor 1) 



Very 
Good 



Good 



1. 



The subject matter relevancy 
to your own interests (pro- 
fessional or otherwise) 



3 



3 



2. The clarity of objectives 



2 



4 



3. The degree to which the objec 
tives and materials provide 
the information necessary to 
accomplish the unit objec- 
tives 



4. The interest value of the 
readings 



5. The assistance of self- 
scored tests to self 
evaluation 



6. The instructional value of 

the exercises (if applicable 5 
to this unit 



7. The instructional value of 
the study guide (s) 



v 
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\ Sunnnary Unit Rating Sheet 

Unit VIII (now X) 
(Instructor 1) 



Very 

Good Good Average Poor 


Very 
Poor 


1. The subject matter relevancy 

to your own interests (pro- 4 
fessional or otherwise) 


2 


1 






2. The clarity of objectives 3 


2 


1 


1 




3« The degree to wnicn cne oojec 
tives and materials provide 
the information necessary to 2 
accomplish the unit objec- 
tived * 


— : 1 


2 


1 




A. The interest value of the 

readings * 


n 

2 


1 






5. The assistance of self- i 
scored tests to self ^ 3 
evaluation 


r 


1 






6. The instructional value of 

the exercises (if applicable 2 
to this unit 


3 


t 






7. The instructional value of 

the study guide (s) 1 


2 


1 
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Summary Unit Rating Sheet 
Unit IX (now XI) 
(Instructor 1) 



Very Very 
Good Good Average Poor Poor 



1. The subject matter relevancy 

to your own interests (pro- 2 
fessional or otherwise) 


4 








2. The clarity of objectives 4 


2 








3. The degree to which the objec- 
tives and materials provide 
the information necessary to 2 
accomplish the unit objec- 
tives 


2 


2 




/ 


4. The interest value of the ^ 
readings 


2 


3 






5. The assistance )of self- 
scored tests to self 2 
evaluation 


1 








6. The instructional value of 
the exercises (if applicable 
to this unit | 


1 








7. The instructional value of 

the study guide (s) 1 


2 
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Summary Unit Rating Sheet 
Unit X (now XII) 
(Instructor 1) 



Very Very 
Good Good Average Poor Poor 



1. The subject matter relevancy 

to your own interests (pro- ^ 
fessional or otherwise) 


2 


1 






2. The clarity of objectives ± 


2 


1 






J • 1 lie UCgLCC LU WI1XL.II LllG UUJCL 

tives and materials provide 
the information necessary to 
accomplish the unit objec- 
tives 


4 








4. The interest value of the 

readings ^ 1 


3 








5. The assistance of self- 
scored tests to self 2 
evaluat iom 










6. The instructional value of 

the exercises (if applicable ^ 
to this unit 


1 


4 


i 




7. The instructional value of 

the study guide (s) 2 











t 
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Summary Unit Rating Sheets 
Unit I (now III) 
(Instructor 2) 



Very Very 
Good Good Average Poor Poor 



1. The subject matter relevancy 

to your own interests (pro- <j 
fessional or otherwise) 




2 


1 


• 


2. The clarity of objectives i 


2 


3 






3. The degree to which the objec- 
tives and materials provide 
the information necessary to 
accomplish the unit objec- 
tives 


2 


< 

i 


2 


1 


4. The interest value of the 
readings 


1 


3 


' 1 


1" 


5. The assistance of self-' 

scored tests to self 1 
evaluation 


1 


2 


1 


1 


6. The instructional value of 
the exercises (if applicable 
to this unit 


2 


1 


.3 




7. The instructional value of 

the study guide (s) 2 


1 


2 




1 
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Summary Unit Rating Sheets 

Unit II (now IV) p 
(Instructor 2) 

) 

Very Very 
Good Good Average Poor Poor 


1. The subject matter relevancy 

to your own interests (pro- 3 
fessional or» otherwise) 


2 


1 




1 


2. The clarity of objectives 3 


2 


2 




1 


3. The degree to which the objec- 
tives and materials provide 
the information ^necessary to 1 
accomplish the unit objec- 
tives 


2 


1 


3 




4. The interest value of the ^ 
readings 


2 


-i 

2 


2 


• 


5. The assistance of self- 
scored tests to self 1 
evaluation 


2 


1 


1 


1 


6. The instructional value of 
the exercises (if applicable 
to this unit . 


5 






1 


7. The instructional value of 
the study guide (s) 


5 






2 
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Summary Unit Rating Sheets 
Unit III (now V) 

v 

(Instructor 2) 



Very Very 
Good Good Average Poor Poor 



1, The subject matter relevancy 
to your own interests (pro- 
fessional or otherwise) 


3 


2 


2 




2. The clarity of objectives 


2 


3 


2 


\ 


3. The degree to which the objec- 
tives and materials provide 
the information necessary to 
accomplish the unit objec- 
tives 


2 


1 


3 


1 


4, The interest value of the 
readings 


2 


1 


2 


-1 


5. The assistance of self- 
scored tests to self 
cvalunt ion 


2 


2 


1 


• 2 


6. The instructional valu^ of 
the exercises (if applicable 
to this unit 


2 


2 


2 


1 


7. The instructional value of 
the study guide(s) 


1 5 









1 
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Summary Unit Rating Sheets 
Unit IV (now VI) 
» (Instructor 2) 



Very Very 
Good Good Average Poor Poor 



1. The subject matter relevancy 

to your own interests (pro- 2 
fessional or otherwise) 






1 




2. The clarity of objectives l 


1 


1 






J, Ine degree co wnicn cne oDjec.^ 
tives and materials provide 
the information necessary to 
accomplish the unit objec- 
tives 


1 


1 


1 




4. The interest value of the 
teadings m 


# 

/ 


2 


1 


1 


5. The assistance of self- 
scored tests to self 
evaluation 


1 


1 


1 




6. The instructional value of 
the exercises (if applicable 
to this unit 






1 




7. The instructional value of 
the study guide(s) 


1 


1 







\ 
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Summary Unit Rating Sheets 
Unit IV (noy VI) 
(Instructor 2) 



Very 
Good 



Good 



Average 



Poor 



Very 
Poor 



2. 



The Subject matter relevancy 
to your own interests (pro- 
fessional or otherwise) 

The clarity of objectives 



The degree to which the objec- 
tives and materials provide 
the information necessary to 
accomplish the unit objec- 
tives 

: > 

The interest value of th^ 
readings 



The assistance of self- 
scored tests to self 
evaluation 



The instructional value of 
the exercises (if applicable 
to this unit 



1 ' 



The instructional value of 
the study guide (s) 
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Summary Unit Rating Sheets 
Unit V (now VII) 
(Instructor 2). 



Very 
Good 



Good 



1. 



The subject matter relevancy 
to your own interests (pro- 
fessional or otherwise) 



2 



2, The clarity of objectives 



2 



1 



u 



3. The degree to which the objec- 
tives and materials provide 
the information necessary to 
accomplish the unit objec- 
tives 

A. The interest value of the 
readings 



5. The assistance of self- 
scored tests to self 1 1 
cvalua t ion 



6. The instructional value of 

the exercises (if applicable 1 
to this unit 



7. The instructional value of 
the study guide (s) 



M 
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Appendix C 

The Students 

Evaluation of their training by the R&D 
students. 



Reports on R & D students by their Supervisors 
and Colleagues. 



I. 
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MEMO TO: Doris T. Gow/^^ 

FROM: Diane J. Davis 

DATE: December 6, 1973 — 

SUBJECT: PIC Twining Materials 

As a student who has completed training in the design and development 
of curriculum through the use of your materials, I would like to share 
with you some qf my impressions concerning those materials. 

I feel that one of the most important characteristics of the curriculum 
is the way in which it is organized. The individual curriculum units have 
enabled me to use these materials extensively as a resource in my continuing 
work' as a curriculum specialist. I am currently employed in this capacity 
at the University External Studies Program and my work *there involves assist- 
ing faculty members in preparing individualized structured materials for use 
by adults studying independently at home. It has been extremely helpful to 
me to be able to' refer to these materials whenever we are faced with a 
particular developmental problem. 

Another aspect of the materials which has been extremely helpful in 
this sense is the use of primary sources in the reading material, It is 
helpful, when preparing any curriculum rationale, to belable to refer to 
these sources in identifying or defending a particular appropriate strategy 
or procedure, for example. 

My own training, through , the use^of these materials, has proven most 
effective for my current responsibilities as a curriculum specialist here 
at the "External Studies Program. I have been able to use my skills in cur- 
riculum development to assist our faculty in designing courses that have lead 
• to student success in this Program. The use of these materials for graduate 

*. . 86 
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Doris T. Gow 

Page 2 t 
December 6, 1973 

students in this Program illustrates the wide variety of both student interests 
and student characteristics for which the materials are applicable. Although 
my own background was not specifically in the field of education, I was abl^ 
to use these training materials successfully to achieve skills in designing 
and developing curricular materials. 



J 




/ 



Nick Laudato 

The effectiveness of the R&D Training Program ca^n best be evaluated 
through an analysis of its constituent parts, and their interactions with & 
one another. For purposes of simplicity, let us view the program as the 
sum of three components: 1) The materials, including the unit materials 
and readings; 2) The interaction sessions including those between the 
students and staff (classroom meetings) and those among students only 
(group activities); and, 3) The practicum and internship experiences. 
Each of these contributed in unique ways to th6 program as a whole. The 
last two components, for example, will invariably differ markedly in fheir 
contributions and value for every group of staff and students and for every 
individual. Conversely, the materials fitfould be transferable to other situ- 
ations with little difficulty. Each component shall thus be examined 
individually. 

The materials were -to a ldfrge extent individualized. In this form, 
they provided an excellent framework from which to pursue the study of, 
both the basic principles of sound curriculum development and also of 
specialized fields of indiyidual interests. Perhaps the most valuable, 
aspect of the materials, however, was the manner in which they were organized 
and in which the content was sequenced. The process of transition from one 
state of knowledge to the, next, more sophisticated state was orderly and 
relatively smooth. There were, of course, at first many frustrating moments 
as enthusiastic students desired to quickly comprehend the intricacies of 
what "was only meant to be an overview of things to come. After attaining 
a more global picture, beginning again in detail was most instructive. In 
retrospect, the manner in which knowledge and skills, were slowly built upon 
cme another was remarkably efficient. Finally, a great'deal'wsS learned from 
observing the manner in which the^materials themselves were developed by the 




staff and from participating in minor ways as a group in an attempt to play 

« 

a more active role in our own education. 

A reciprocal relationship seamed to exist between the materials and 
the interaction sessions. For one, the classroom and group meetings certainly 
enriched the materials greatly. It seems, however, that the quality of the 
materials and their organization enabled the meetings to be as effective as 
they Were by providing a solid, informative basis for discussion and debate. 
Basically, the interaction sessions provided a format In which the validity 
of newly acquired ideas and concepts could be tested and organized into 
cohesive statements of (educational philosophy and method. The interaction 
session, along with th » internship experience, also allowed for previously 
acquired knowledge to >e placed in a more realistic context. 

The internship experience was something of a crowning touch. It was 
tation to months of preparation: The chance to test 
It provided the essential opportunity to synthesize 
previously learned knowledge, concepts and skills along with those necessarily 
acquired in the particular, specialized internship. The internship also 
provided possibly the iest test of the materials. In this respect, I believe 



the only logical culmi 
our abilities "on-line 



rials were 8uc4essful. 



the mptei 

V 

In conclusion, I consider the most interesting and rewarding components 

\. 

to be the interaction sessions (especially group) and the internship, but 
the most essential to be the materials themselves. All three components 
were indispensible in making the program a most successful enterprise. 
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MEMO TO: Dr. Doris Gow 

FROM: S. Claire McCormick * 

DATE: December 10, 1973 

SUBJECT: Report on Experiences in Program 2: Curriculum Development 



Program 2 (The Curriculum Design Course) answered a need which I had 
felt in trying to initiate change toward a more individualized learning envir- 
onment in two different small high schools. Implementing new forms of 
Scheduling was comparatively easy to manage, but assisting teachers in planning 
really individualized instructional sequences was a challenge 1 felt I was 
not quite meeting. It was gratifying to me "to find a program which met this 
need, and also took me beyond into the area of invention and design of new 
forms of curriculum. 

•JJiere are three chief benefits which I gained from participating in the 
first year's training. First, the materials presented in the individual 
learning packages, taken as a whole, integrated new knowledge derived both 
from educational psychology and educational research as this knowledge 
applies to curriculum design and development. The systems model of curriculum 
design developed in this program is invaluable as a new way of looking at 
curriculum. 

Second, the manner in which the units were constructed and the types 
of exercises included forced me into a more analytical way of thinking than 
I had been accustomed to use. This was good for me. 
* Third, the experience of doing individual projects and group projects 

was also good in that it encouraged independent exploration of a problem. 
I could not have produced my internship project without the prior training 
afforded by our program in curriculum development. 

no 
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Doris Gow 
Page 2 

December 10, 1973 



Finally, I believe that our instructors who designed this program are t< 
be commended for pioneering in the development of learning packages on the 
graduate level. Also noteworthy is the fact that we as students were exper- 
iencing the kind of learning situation for % which we were being trained to 
design and develop materials. This program hap served to further define an 
area which is beginning to emerge as a discipline itself; namely, the tech- 
nology of curriculum design. 



t 
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T. Whiting 

My R & D Training 

Before coming to the University of Pittsburgh as a graduate student in 

the R&D Training Program, I had been working as an educational coordinator 

in a program where one of my major responsibilities was that of designing 

and implementing a curriculum foj high school dropouts. I came to that job 

with only the experience of a secondary social studies teacher. I had had 

no training or experience in curriculum development. During that year, I 

examined many curricula designed for secondary students and based on the 

needs of the students in the program, I put together bits and pieces from 

these curricula and made a new one. To my amazement this "cut and paste" 

curriculum actually worked for those students. However, I was dissatisfied 

with this method of curriculum making and was really interested in learning 

i 

how to design curricula. The R&D Training Program offered me the oppor- 
tunity to acquire this skill. 

A systems approach to curriculum development was offered to the students 
in the R&D Program. The significance of this approach was the idea that 
curricula should be designed and developed based on specific instructional 
objectives which, when met, can be objectively evaluated to measure their 
instructional effectiveness. Through pre-teeting the student can enter the 
curriculum at his present level of competency, and, if the instructional 

methods are ineffective, feedback is provided into thp system to help 
t 

re-evaluate &nd correct the instructional process. This approach no longer 
places the responsibility for learning or "mislearning" on' the student but 
places the responsibility on the instructional material(s) designed to 
translate the objectives to the student. 

Over^he past 2 1/2 years, the R&D Training Program has provided me 



with many valuable educational experiences. Th$ most valuable experience was 
that of working as an intern with Dr. D. Gow on an experimental social^ studies 
curriculum for third graders in Frick School. Frick is an inner-city elemen- 
tary school and my interest was focused on curriculum development in the 
affective domain for the urban elementary^school child. Dr. Gow's social 

J 

studies curriculum encompassed both the cognit^e and the affective domain, 
utilizing social science concepts and methodology to approach the affective 
area of learning. The internship provided me with the opportunity to apply 
many of the theories and skills taught in the first year of the program, and 
to observe first hand the results of their use. Working on the social studies 
curriculum also allowed me to practice my belief that curricula should not 
be developed in a vacuum, but with the designer in clotfe contact with the 
faculty and students for whom the curriculum is being designed. The design 
and development of a curriculum is at times a slow process which requires 
patience and I learned to be more patient and not to expect everything to 
fall into place v at once. Since I was working in an urban elementary school, 
I al^o had to learn how to cope with many of the problems which arise in 
such a system. But, the greatest reward came when observing how well some- 
thing worked when it -was used to teach® a concept to a small child, something 
that you had helped to create. Curriculum development is at best a skill 
which requires time, practice, patience and ingenuity to acquire. The R&D 
Training Program provided me with the opportunity to begin to acquire and 
practice this skill and it also gave me the incentive to develop more ideas 
for futher research into the area of curriculum development in the affective 
domain. 
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lo - Doris T. Gow 

p 
/ 

i-kom: Samuel Deep, Director, University External Studies Program \J £Lyi%^ 
■^a 1 1 : December 5, -1973 
suuja.i : 



\ 

Diane Davis, a student from the L.R.D.C. Training Program in 
Curriculum Development, has worked as a curriculum and instructional 
specialist with this Program since its initiation in the Fall of 1972. 
As Director of the Program since February of this year, I have been 
able to observe artd evaluate her work here in terms of the effectiveness 
of her training for the type of curriculum development required by this 
Program. 

The University External Studies Program provides college level 
courses based on individualized .structured materials, for independent 
study by adult students who are unable to attend regularly ^efteduled 
University classes due to such things as geographical location, or job 
and family responsibilities. The curriculum and instructional specialists 
in this Program are required to work with faculty members to assist them in 
designing instructional materials which are appropriate and effective for 
the students we serve. This wor\ requires such skills as: 

« 

Analysis of Cours Material in three areas: 

-Component Analysis 
-Content Analysis 
-Concept Analysis 

Evaluation of appropriate instructional strategies 

' \ 

Coordination of Development of Goals/ Objectives, 
Text;, and Testing Materials Q^th diagnostic and 
evaluative) 

Preparation of Instructional Strategies 
for On-Campus Work-Shops ^ 

Analyze course evaluation .and develop strategies 
for change when considering revision. 

Analyze changing studentlneeds as th^y apply to 
course development. 

Analyie professional skills using^ feedback from: 
-faculty 

-staff colleagues 
-student^ evaluation of courses 

ERIC * '04 



Continuing exploration of new instructional strategies 
as they apply to curriculum development. 

Assist students who have problems with instructional 
materials . 

Diane's formal training in curriculum has come solely through 
the L.R.D.C. training Program and the PIC model. This training has 
proven extremely effective for meeting the instructional goals of this 
Program. 



\ 
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I Al;ULTY OY AKIN AM) SCU;NC;lS 
UNIVLUSITY OI IMXrSHUlUill 
PITTSBURGH, PENNSYLVANIA 15213 



OFFICE OF THE DtAN ' 

25 May 1973 



Dr. John L. Morgan 
LRDC 

8th floor, LIS Building 
Dear Dr. Morgan: 

I am pleased to 



jite 1 to^you about Diane Davifcyj 

Ms. Davis hai helped me injmany ways in the ! | 
several External Studied courses that I have taught. 
Her Ideas about? curricujjum, fctudy guides, and arrangej- 
ment of materiajls have q^richedj my courses much. She 
has been most cpoperati|e in doing a wide £ange of 
assignments in ^ustainijfig a meaningful relationship 
with the students. Witlhdut her help, the burden of 
handling these courses 4 as well as their effectiveness 
would have teem lessened Materially. 



Sincerely, 




. - v ******* 

Slesingery 



kiben E. 
Associate Dean and/ 
Profeseor x>£ Eeon'dmics 



RES: mo 
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M E M 0 R A N D U M 



TO: John Morgan 

FROM: JafDes Holland 

DATE: 8 June 1973 

SUDJECT: Recommendation for Diane Davis 



Last year Diane carried out an independant research project with me. 
The project required the implementation of our prc-school skills v.'ork 
in Eriok School. This p,ave ne nn opportunity to observe Diane's skill 
in an unusually vide ranp,e of attihutfrs. She had to provide liaison 
betveen the lab and teachers in the school, had contact vith the 
students, had to systematically collect data, take part in conferences 
with the school administration, LRDC faculty, write tip a careful^ * 
evaluation of all problems found in implementing the program. Diane 
proveyd to be a thoroughly capal)lc professional in all aspects of her 
vork. She showed p,£eat responsibility and initiative in handlinr^fchis 
project. She proved to be a sensitive^ critical observer and keptr 
thorough protocols; she was able to handle the rather difficult problem 
of working with so many different pggple havinq so many different 
objectives. All in all/ I am thoroughly confident in Daine's ability* 
to make important contributions to educational development. She is 
also a pleasure to be around and the kind of person I would welcome as 
a coll^^ue. 

JGH:rh 
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LEARNING RESEARCH AND DEVELOPMENT CENTER 

UNIVERSITY OF PITTSBURGH * PITTSBURGH • PENNSYLVANIA 15213 



DIRECTOR 208 M I BUILDING 
LABORATORIES: M. 1 BUILDINCj 
ADMINISTRATION 160 NORTH CRAIG STREET 



PHO 

624-4884 



MEMO 



\ 

TO: Doris Gow 
FROM:- Dick Roman 
DATE : December 3, 1973 
RE: Nicholas Laudato f s Training 



Nicholas Charles Laudato interned with the Computer Assisted 
Instruction in Problem Solving Project after he was trained. At the 
time he joined us he had no previous computer experience and rio direct 
instructional experience except what he had learned in the R fi D pro- 
/ gram. The specific skill required to write and evaluate computerized 
instruction differ in many particulars from those used in more tradi- 
tional curriculum, and someone trained specifically for other media . 
often can not make the transition easily. Nick's training however 
was immediately transferred to the new situation; I believe that 
speaks highly for the R 5 D program as well as for Nick himself. 



ERLC 



The specific skills Nick brought with him include: 

the ability to observe and describe the actions^ 
students took on a computer prog^m. He. was able 
-to separate what he saw from what he inferred. 



c 



Wl 



the ability to diagnose difficulties in lessons <: 
accurately and make moderate interventions with the 
students to correct the difficulty. 

the abilj.ty^to <Jo task analysis* on specific objectives 
and design curriculum that fit the constraints of the 
available instructional paradigms. 

the ability to generate several .alternative solutions 
to an instructional problem and to choose the best 
alternative to achieve his goals. ' 4 

I believe that the year of training Nick received provided him % 
th extremely useful skills and concepts for work in the area of 



curriculum development 

RAR/met Cj g 
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LEARNING RESEARCH AND DEVELOPMENT CENTER 

UNIVERSITY OF PITTSBURGH • PITTSBURGH • PENNSYLVANIA 15260 



TO: Doris Cow 
FROM:/ Isabel Bectf 

DATE: December 4, 1973 

/ 

RE: Sister Claire McCormick's Internship with the Reading Project 

Sister Claire McCormick was an intern with the reading Project of 
LRDC during the fall and winter terms 1972-73. Ufae LRDC Reading Project 
is currently involved in the development of an individualizgd-adaptive 
reading system for the primary grades, known as the New Primary Grades 
Reading System (NRS). Sister McCormick became associated with the Reading- 
Project when approximately one-third of the new reading system had been 
developed. She quickly demonstrated that she was extremely well-versed 
in many aspects of instructional design. 

As a summer project prior to becoming an intern, Sister tycCormick 
prepared an indepth analysis p£ some of the instructional materials 
contained within certain leveUs of NRS . She analyze^ the NRS materials 
for strategies of ins tructionbased in research. In order to carry out 
the analysis, Sister McCormick developed a model for evaluating the audio , 
lessons which included: 1) inducing the behavioral objectives from the 
instructional materials, 2) inducing the cognitive -skills required to . * 

respond to the materials, and 3) analyzing the individual frames or groups 
of frames for stimulus, prompts, response, and management elements. The 
purpose of the analysis was to provide the evaluator of .curriculum 
materials with enough information to determine the quality and intensity 
of learner interaction required by the materials. 

As an intern, Sister McCormick became involved at various times with 
almost all the. aspects of thr continuing development of NRS. "She designed A 
cassette response pages and wrote ^fche accompany/Lng audio scripts; she 
wrote independent seatwork which follows the djassette lessons; she wrote 
read-^alone stories and designed games, "the content of which corresponds 
to specific instructional levels of NRS; she observed NRS being used in 
developmental classrooms and taught certain lessons to the children- in 
these classrooms. And, like all Reading Department staff members, she 
assisted with anything that needed to be done to facilitate the develop- 
ment of the program, such as proofreading copy of the children's workbook 
pages and checking audio cassettes. Sister McCormick was able to fit her 
individual assignments into the perspective of the development of the 
total reading system. 

It was clear from the results demonstrated while she was an intern 
that Sistert McCormick had received excellent training in curriculum design 
and development, and that she was we Ll-fversed in the theories of instruction. 
Her work on the Reading Project showed that she could translate the theories 
she had learned into the development of an actual program. 
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A description of a portion of Diane Davis ' 
work as an intern at the University External 
Studies PrograSh. This was presented by Diane 
a*t the AERA Meeting in New Orleans, Feb., 
1973. N. 
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THE DEVELOPMENT OF AN INSTRUCTIONAL DELIVERY SYSTEM FOR 
EXTERNAL STUDIES IN HIGHER EDUCATION 

* John L. Yisager 6 Diane J. Davis 

Background 

The estabiishment of non-resident educational programs in higher educa- 
tion has been receiving an increasing amount of attention in an effort to meet 
^ the diverse educational needs of the American public. Although a number of 
prototype systems exist that provide such instruction, there have been few- 
attempts, in this country, to develop' non-resident educational systems that in- 
clude both a flexible delivery mode and quality. instruction. The primary re- 
quisites in the development of a non-resident educational program are: (1) that 
a delivery system be developed that has the characteristics of loW £ost, wide- 
area distribution and be available to the student on demand, T2) that the pro- 
gram insures that the content beipg transmitted is of high academic quality, 
(3) that the program provides for student motivation, and (4) that alternative 
learning modes be made available to accommodate individual student learning 
preferences. 

Vhe University External Studies Program (UESP) at the University of 
Pittsburgh began with the assumption that there existed, in Western Pennsylvania*, 
certain individuals who, had not been afforded access to higher educational 
opportunities. The purpose of the External Studies Program is to provide ex- 
tended educational opportunities to those segments of the population that are 
currently denied access to undergraduate and graduate instruction because of 
geographical and situational factors. That is, there are at present a number 
of individuals who cannot avail themselves of the opportunity to^pursue higher 
education because of such factors as commuting distance to and from an insti- 
tution, family responsibilities, physical handicaps, and Work situations such 
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as those entailing irregular schedules. In addition, there arc individuals who 
wish to change positions or careers and desire to obtain new credentials or ^to 
further their education and yet must simultaneously maintain and support them- 
selves or other individuals,. 

Central to the development of a program designed to meet these indivi- 
dual educational needs, is the specification of a delivery system for the 
dissemination of instruction. This delivery "system" consists of the instruc- 
tional delivery model and the instructional support systems. The "instructional 
delivery model," then, refers to the curricular methods, mode, and media chosen 
for transmitting the course content. There is an interacting relationship 
between the characteristics of the instructional delivery system and the goals 
of the external program itself. That is, a number of specific assumptions are 
derived from the philosophy and goals of an external study program that act to 
limit and define the parameters of the delivery system itself. 

1. The system must be flexible in terms of its 
accessibility and state of "readiness" or 
availability to the learner at any given point 
in time. 

2. The system must permit the student to assume 

a great deal of independence and responsibility 
for pursuing specific learning goals. 

3. The system must provide instruction that is 
adaptive to the individual needs of* the learner. 

Each of the delivery modes described below, combined with the support 
systems discussed later, constitute a separate type of delivery system that 
the UESP Program attempted to implement and examine. 

It is within this context that the University External Studies Program 
recruited faculty members who, together with the Program staff, developed and 
offered five external studies courses which were the equivalent of on-campus 
courses and which represented four alternative instructional delivery modes. 
Each system served as one means for attempting to meet the diverse instruc- 
tional needs of the UESP students. The Program was able to obtain background 
data for 121 of the 153 who enrolled in UESP courses during the initial offer- 
ing. An analysis of this background information shows that these students 
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ranged in age from 19 to 55 and the majority did have employment or family re- 
sponsibilitips which prevented them from attending regularly scheduled on- 
campus classes. The reasons, as the students listed them, for preferring this 
type of study to traditional classes included the following: family responsi- 
bilities, work schedules, time conflicts with regular classes, and travel incon- 
venience. Less frequently mentioned were health and parking. 

" ' J 
THE DELIVERY SYSTEMS 

General Characteristics 

One of the primary goals of this External Studies Program was to develop 
a system which was easily accessible to the student at any point in time. It 
was for this reason that written, self-instructional materials were chosea as 
the primary medium for the delivery of external studies courses to these stu- 
dents. Written materials have the advantages of being readily accessible to 
the student at any time and of being adaptable to the individual needs of the 
learner as well as to a wide variet^f content. In addition, they are rela- 
tively inexpensive when compared to the other media and can be used indepen- 
dently from any other media. ^ 

For this Program, the individual faculty members were requested to de- 
velop self- instructional materials or to select from written materials already 
available in their field. They were assisted by curriculum specialists from 
the UESP staff and all materials were extensively reviewed before being dis- 
tributed to the external studies students participating in the Program. The 
four types of delivery models developed are" as follows: 

1. Television with Supplementary Materials ^ 

2. Programmed Instruction with Supplementary 
Materials and Readings 

3. Reading Lists with Summaries and Supplementary 
Materials 

4. Structured Curriculum based upon a Process Model 
for the Individualization of Curricula 

In addition to the curricular packages, based on each of the four 

delivery models, each system included the following supplemental support systems 
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An Advisement-Counseling System . The function of this system was 
provide the student with advisement and counseling 'personnel who could assist 
them with personal or pfoijessional counseling needs. The main portion of this 
service was provided through the staff of the School of General Studies which 
included personnel specifically trained for these purposes- In addition, the 
UESP system operated primarily in providing the following kinds of supportive 
assistance: 

a. Information concerning UESP courses and the award of 
formal college credit. 

b. Procedures for registration for UESP courses and assis- 
tance when special registration problems arose. 

c. Assistance with difficulties in achieving course objec- 
tives and procedures for extending time limits to meet 
external pressures; encouragement when students encoun- 
tered problems. f 

d. Providing a student manual containing infajran^tien on 
how to study independently, as well as a map of the Uni- 
versity indicating special resource areas and a list of 
courses to be offered the following term. 

A Communications Support System . It was recognized by the UESP staff 
that since the student was primarily engaged in independent study, there was 
a pressing need to provide alternative ways by^whlch the student would have 
access to the faculty and Program staff. The communications support system 
provided the student with the following means for communication: 

a. Telephone communication - Students were given specific 
numbers to dial in order to reach the course instructors, 
the UESP staff, or specifically appointed teaching assis- 
tants who could assist them. 



b. . Student mailgjfc - The students were provided with self- 

addressed forrifc which they could use for mailing in ques- 
tions to the course instructor. There was space for the 
instructor -^e^ponse- and students received an answer in as 
little time as possible. . 

c. Newsletter - Students received a newsletter four times 
during the term informing them of special dates and 
events which they should be aware of or which were rele- 
vant to their course interests. 
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d. Special memos ^ Special memos were prepared and mailed by 
the UESP staff when an instructor wished to make a special 
announcement to his students. 

A telephone answering service, originally scheduled to be installed in 
time for the first terra, was not delivered until the end of that term^and had 

to be postponed for use* in the following term. 

* 

Instructional Interaction Sessions , In order for students to be able 
to meet and interact with the course instructor and with the others^tudents in 
the course, three interaction sessions were held during the term. These 
sessions were held on campus on designated Saturdays. The sessions were used 
primarily for lectures, group learning activities, formal testing, and to pro- 
vide an opportunity for the student to ask questions and to get to know the 
other students. Program staff members assisted the instructors when necessary, 
serving as proctors or discussion leaders. 

When a student could not attend one of these interaction sessions, 
as appointment was made with. the instructor to enable the student to make up 
any test missed, and to provide him/her with relevant information discussed 
at the session. Special reports on the interaction sessions were sent to 
absentees. 

Specific Characteristics of the Four Types of Instructional Delivery Models 

Each of the four instructional packages had some type of written 
materials ranging from reading lists and summaries to highly-structured ex- 
plicit learning modules. At the same time, each contained components that 
were unique. The following is a brief description of each delivery model 
and its packaging forma t, (See Table 1 for components of the various 
packages . ) 

Type I* Television with Supplementary Materials . One of the five 
initial courses, Law and Morality, was composed of televised lectures and 
presentations offered through the Sunrise Semester along with supplementary 
books and instructor-written lectures which were designed to assure that the 
student was presented with all the materials necessary in order for him to 
meet the course objectives. 
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The student was presented with material initially designed to be 
supplementary to the televised presentations. During the. design stages of 
the course, it was expected that the production center would forward detailed 
information concerning the topics and information to be presented on the 
televised lecture series, as this information had been requested during the 
initial stage of development . The information actually received, however, 
was less than adequate for use as a basis for the design and development of 
a well -organized course in Law and Morality and it was soon realized that the 
televised lectures Would have to serve as supplemental content and that the 
instructor would have, to develop and 'present materials- which would provide the 
student with tit e information necessary for meeting the objectives of the course 
The Sunrise Semester lectures did, however, provide the student with another 
frame of reference for the course 'and offered contact with the general content 



area. 



Type II - Programmed Instruction with Supplementar y Readings and 
Materials . A second type of delivery model, represented in the Introduction 
to Economics course, was based upon a programmed textbook with supplementary 
readings and materials. This supplementary reading consisted of a textbook 
which accompanied the programmed text. There were no instructor written lec- 
tures specifically designed to meet the stated course objectives. 

This course depended almost entirely upon previously compiled textbooks 
and required a minimum of developmental time for the instructor since only a 
few instructor "handouts" were prepared. The instructor's greatest task, in 
this case, was to organize specific assignments and statements of objectives. 

Type III - Reading Lists with Summaries 6 Supplementary Materials . A 
third type of model was pilot tested in a History course entitled "The Decline 
and Fall of Modem Europe," offered by the UESP Program. This course consisted 
of thirteen books which the students were required to read, plus instructor- 
written summaries and study questions. The answers to these questions were 
not mailed in by the students and were not self-graded so that students re- 
ceived very little feedback orf*their*p^ogress, other than that received at 
the interaction sessions, befire being required to Jake the final examination. 
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. Tyfre IV , ^Structured Currictilnr Materials Based Upo n a Process Model 
for the Individualization of Curricula . This course in Curriculum and Super- 
vision*^ based upon a process model for the design and development of curri- 
culum developed at the Learning Research and Development Center at the Uni- 
versity of Pittsburgh. This- is a highly developed research-based model repre- 
senting a process for individualizing curricula through structured materials. 

This is the most highly structured of the courses offered as it is / 
based upon a specified model for curriculum development- It is very explicit 
in terms of student objectives and student procedures. The course provides a 
pretest for identifying the knowledge which the student may already have, and 
offers alternatives within the curriculum for meeting the individual goals of 
the student; for example, students are directed to certain readings if they 
are not interested in that particular area. The curriculum attempts, first, 
to provide the student with the knowledge and skills necessary for branching 
out into specific areas, and then to provide alternative content so that' he 
can apply the "knowledge and skills to his own field of interest. In addition, 
the student is provided with a means for evaluating his own progress through 
curriculum embedded testing for which answer keys or response criteria are 
provided which enable him to evaluate his own responses to the criterion 
referenced test items. 
Comparing the Four Delivery Types 

Although it is difficult to attempt to compare these different delivery 
models, since they are within different content areas and are designed and 
offered by different faculty members, one can examine the similarities or 
differences among the various types of presentation. As mentioned earlier, 
all four had a written instructional component. Type I incorporated another 
kind of medium— television. The major difference, however, among the various 
kinds of delivery was the degree of structure and specificity. While, the 
History course, Type III, offered minimal structure in presenting the student 
with a book lis^and some general direction, the Curriculum and Supervision 
course presented the student with specific objectives and step-by-step pro- 
cedures for achieving those objectives. The televised Law and Morality course 
and the programmed Economics course offered some degree of structure- more 
than the History course but less than the CSS course. 
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. The only other major difference among the four instructional delivery 
models is the amount of self- evaluation provided for in the materials The 
CSS course. Type IV, contains self-scored curriculum embedded test items as 
well as self-scored posttests. None of the other courses presented the stu- 
dent with so much opportunity to evaluate his own progress throughout the 
course. The Law and Morality course. Type I, did require the student to mail 
in answers for case studies presented within the units, but these were faculty 
graded and served as a parjiLl basis for the student's final grade. 

Another mean's for comparing the courses might be to examine the student 
performance for each of the instructional delivery systems as a whole. (See 
Table 2 for student grades.) 

, TABLE 2 

; Final Grades for Four U^SP Courses 



1 Course 


A 


B 


c 


D 


F 


G* 


Total 


Law & Morality (Type I) 


3 


3 


2 


0 


v 0 


7 


-15 


Economics (Type II) 


6 


13 


1 


0 


0 


1 


21 


History (Type III) 


2 


5 


9 


0 


0 


1 


17 


Curriculum Supervision 
(Type IV) 


10 


4 


3 


0 


0 


8 


25 


TOTAL 


21 


25 


15 


0 


0 


17 


78 



*The "G" grade represents an incomplete grade and was administered by mutual 
agreement between student and^ instructor to allow the student additional time 
to complete the course work. This was an alternative to allow for student 
time flexibility based upon his individual needs and responsibilities. ^ 

: V— *-f ■ 

Student Reaction 

Generally, the student reaction to all four types of instructional 
delivery were positive, with all but one student (a History student) indi- 
cating that they wouH be interested in taking another UESP course if it were 
offered in their field of interest v 

' It can be noted that many of the students in the Type I, II, and 11 
courses suggested the addition of components which would lend a greater 



111 

25, 



degree of structure and direction to these courses. Statements such as "Not 
knowing what points the instructor is stressing," "more sessions to help me 
correlate the ideas and information into a pattern," "1 feel we need more 
direction," etc^) indicate that these students were somewhat confused as to the 
direction- in which they were supposed to progress with . the Materials . In 
courses which did not offer instructor-written materials (Type II especially) ^ 
many students caim^jSted on the need for this type of communication. Since 
these students do not have the opportunity for classroom lecture^, where they 
could pick up such things as instructor emphases and points of special interest, 
this information seems to be .greatly appreciated within the course materials. 
These kinds of cues assist the student in identifying learning objectives and 
testing points and are in that sense an important learning aid. 

Many of the students indicated that a major advantage to this type of 
study was that they could study when and where they wished and they always had 
the materials on hand for review when necessary . \Jhe Law and Morality (Type 
I) students, however, did not have tl\is convenience with the televised lectures 
since they were presented only once and there was no alternative time schedule. 
This proved to be disturbing and inconvenient to some of the students as indi- 
cated by comments on their course evaluation sheets.* 

Faculty Reaction - * 

As a rule, the individual faculty members seemed confident that the 
students had met the course goals (implicit or explicit), although' they recog- 
nized weaknesses within their own types of instructional delivery, the History 
professor, for example, noticed that his students seemed to request additional 
direction and at one point commented that he had been asked to hold an addi- 
tional interaction session to assist the students. All of the faculty mem- 
bers seemed to recognize the value of some of the structured components for 
the packaged materials. They recognized that, for the external studies student, 
a certain amounfof direction must be present within the materials since they 
are required to meet certain course goals. 



*For purposes of program evaluation, the student was asked to till out course 
rating sheets (2). as well as unit rating sheets and background information 
forms . 
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Resources 

In examining these four alternative formats, it is necessary that 
attention be given to the management' effort and costs expended in developing 
and offering each course. Although it is not possible to specify the "real" 
cost of each course, anas^essment can be made which indicates the relative 
cost of each pourse in terms of UESP costs , faculty effort, commercial develop- 
ment cost and student costs. These relative costs are presented in Table 3. 
\ 

v - TABLE 3 ' 
Reiajtiye Distribution of Costs by Course ^ 

^ ^\Faculty Student . 
Course UESP Effort Commercial Costs 

Law h Morality (Type I) Moderate High High Low 

Economics (Type 2) Low Low Moderate Low 

History (Ty^e 3) Moderate Moderate High High 

CurriculOfli \ Supervision High - High Low Low 

(T#e 4) 

3^ UESP co^ts refer to those costs that are associated with providing 
chnical developmental assistance to the faculty, production costs, and the 
general administrative support required to coordinate and manage the course. 
Since the Curriculum Design course based on the structured-curriculum model 
involved the greatest volume of technical material, it required the greatest 
amount of UESP resources. At the other extreme, the utilization of commer- 
cially available programmed instructional materials and supplemental text- 
books required only a minimal amount of UESP resources. The amount of faculty 
effort required to develop the individual courses corresponds to the same dis- 
tribution as that of UESP costs. The structured-curriculum coijrse required 
an extensive amount of faculty time equivalent to approximately four or five 
man months of development. Commercial development costs refer to the rela- 
tive commercial cost that have to be expended to develop the materials.' 
Naturally, the courses utilizing television and textbook supplements are very 
expensive because of the television production costs. Also, the course 
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requiring thirteen textbooks had high commercial development c^sts associated 
with it. These commercial development costs are important to recognize since, 
if such developmental costs were nqt assumed by another agency, they would 
have to be assumed by the Program. Jn spmo instances, this could be very pfb-_ 
h^bitive. Finally, it is important to consider the cost to the st'udent in 
acquiring the necessary instructional materials. Since the structured- 
curriculum model, the programmed instruction, arid television modes of presen- 
tation were either internally produced or else required dYily a limited num- 
ber of commercially available materials, these student costs were relatively 
low. The lengthly reading list required by the History course resulted in 
about 50% more student costs. 

In summary, based on th<? cost factors associated with the development 
of these materials, attention should be given, when possible, to the utiliza- 
tion of commercially developed materials, particularly of the. programmed 

instruction mode. In terms of UESP, the structured-curriculum model has been 

< 

selected as the primary 'format mode. 



1** 

Summary 



The advantages in flexibility and accessibility of written instruc-- ^ 
tional materials was re-emphasized throughout this attempt to try the various 
/ypes of instructional delivery systems, as were the limitations and possi- 
bilities of television as an instructional medium. A value, in terms' of ex- ^ 
plicit student direction and Student self-evaluation, was recognized in the 
highly-structured curriculum. It seems apparent throughout this study that written 
materials can provide the external studies student with the information needed 
for meeting the objectives of a course and that these materials can be supple- 
mented with other media or can be used exclusively by the student. It was also 
recognized that, in order to address individual student needs and provide 
instructional alternatives to meet those needs, this Program should continue to 
experiment with various types of packaging and with various media. 

It can' be noted here that the structured-curriculum process (Type IV) 
is recognized to be extremely adaptable to this kind of experimentation since 
it provides a process for identifying student needs (in terms of a learning 



114 

O 28 

ERIC <v 



« 



hierarchy and structured pretest) as well as a process for identifying appro- 
priate use of media based upon the characteristics of the student population, 
and the type of objective to be achieved. This kind of instructional delivery 
system, then, is seeff to be extremely useful in maintaining studenf direction, 
specificity, and accessibility while at the same time allowing for the indivi- 
duality and creativity of the instructor in attempting to make use of other 
media and methods in his course design. 
Future Developments 

Although the University External Studies Program has achieved some 
degree of success during its initial stage of operation, the staff of this 
Program feel that alternative instructional formats need to be considered in 
order to provide a comprehensive learning system that will most- adequately 
meet the needs of its students. At the. present time, the structured- curriculum 
model is viewed by the staff as having a high degree of potential for formatting 
the materials, however, the emphasis on paper/pencil presentation needs to be 

further exai^ned. 

In order to systematically examine additional delivery modes, two 
studies are currently under consideration. The first of these utilizes audio 
cassettes. Because audio cassettes are readily available at a reasonable cost, 
the use of them seems to be of some merit. The audio cassette would permit the 
student the option to hear a discussion representing various points of view 
about a given .topic, to have faculty summaries prepared on given aspects of 
the course, or could be used as a communication device between faculty and 
students in terms of answering a student's specific questions. At the pre- 
sent time, audio cassettes are being considered as supplemental components 
of the instructional package and not as the primary program. 

Another system also being considered is that of cable casting; 
{hat is, the use of br6adcasting systems via a cable television network. On 
the assumption that the growth of this industry will continue at the same 
rate it has over the past five years, it appeals that a network of cable sta- 
tions could provide low cost distribution to a large number of, households. 
This program would involve both tip written materials already prepared by 
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UESP and cable casting. The instructional package would consist of a paper/ 
pencil instructional workbook /based on the structured-curriculum model, three 
interaction sessions per term, and fifteen one-hour television presentations 
per term. Each television presentation would be presented three times in a 
given day--once a week; morning, afternoon, and evening. Therefore, the stu- 
dent would have available three alternative instructional learning modes: the 
interaction sessions, the televised presentation, and the paper/pencil presen- 
tation. Cable casting would permit flexibility through offering three pre- 
sentations of a given lesson at various times in the day and would provide a 
motivational factor in that there would be an implicit pacing for the student 
because of the scheduled television presentations . In addition, it would be 
possible to reach a larger number of individuals who cannot currently utilize 
present educational opportunities. 

It is hoped that by systematically examining various types of alterna- 
tive formats in instructional delivery systems, it will be possible to devise 
a comprehensive instructional system that can be utilized to meet the needs of 
non-resident students. 
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A Computer Word Problem Program designed 
by Nicholas Laudato during his 'intern- 
ship. This is to be published by the 
Learning Research and Development Center. 
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Why ^word problems? I hope that I shall shock a few 
people in asserting that the most important single task of mathe- 
matical instruction in the secondary schools is to teach the setting 
up of equations to solve word problems. Yet there is a strong argument 
in favor of this opinion ... In solving a word problem by setting up 
equations, the student translates a real situation into mathematical 
terms: he has an opportunity to experience that mathematical concepts 
may be related to reality, but such relations must be carefully 
worked out. Here is the first opportunity afforded by the curriculum 
for this basic experience. 

-Polya: Mathematical Discovery 
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INTRODUCTION 

The area of arithmetic and algebraic word problems generally 
presents the elementary school student with his first experience in 
utilizing mathematics as a tool to solve "real" or physical problems. 
It is an area which is crucial toljjft students' later mathematical 
development and thus comprises an important part of the mathematics 
curriculum. Word problems have, however, continually frustrated 
educators' efforts %p effectively teach competency in them. This 
paper describes an attempt to deal with word problems in the elementary 
school. 

The Word Problem Program has been undertaken as one of 
several prbgrams developed to investigate Computer Assisted Instruction 
in Problem Solving as part of the Oakleaf Small Computer Project. ' 
The Oakleaf Small Computer Project was initiated by LRDC in 1969 under 
sponsorship of the National Science Foundation. Us principle 
purpose was to "...investigate appropriate and effective uses of 
the computer in an individualized school in order to foster the 
adoption of individualized systems of elementary education ..." 
(Glaser and Cooley, 1972). The Word Problem Program is also -related 
to IPI (Individually Prescribed Instruction) Math in that one of 
its goals is to complement the LPI treatment of arithmetic word 
problems. The fundamental goal of the Word Problem Program, however, 
as with all other programs in the Computer Assisted Instruction in 
Problem Solving Project, is to teach elementary school students some 
generalizable problem solvinr skills. This goal will be discussed in 
detail in section 1. 

This paper considers the literature pertaining to word 
problems in both the traditional and computer based modes. On the 
basis of relevant research, theory, and intuition, the paper then 
proposes some instructional strategies and methods, as well as a format 
for a computer program to teach the process of solving word problems. 

120 

34 



J 



the paper also hypothesizes relationships between some variables present 
in word problems and the difficulty of the word problems. Finally, 
the paper concludes with some sample protocols of an experimental 
version of the Word Problem Program in operation. 
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OBJECTIVES OF THE PROGRAM 

The fundamental goal of the Word Problem Program is to teach students 
who use it skills which will make them more effective problem solvers. 
All of the instructional objectives of the program are derived from 
this goal, and specific instructional decisions are made on the basis of 
it. Thus, the program is concerned with teaching not only the skills 
involved' in solving arithmetic word problems, but also skills applica(^e 
to a wider range of problem solving situations. This section delineates 
the cognitive objectives of the program dealing with 'both of these areas 
and also describes the affective goals of the program. Before proceeding 
with this task, however, we must fir3t clarify the meanings we attach 
to the terms "problem solving" and "word problems". 

Several educators have made hypotheses and conclusions regarding 
the nature of problem solving. It may prove enlightening to consider 
some of these here. In general terms, John Dewey described a problem 
as anything that perplexes and challenges the .mind so that it makes 
belief uncertain. Kramer (1966) expands on that basic definition and 
specifies that t*he individual who is facecf with a problem must "analyze 
the situation, gather facts that point toward a solution, decide which of 
those facts are pertinent to the problem, artd then, by reasoning 
logically with these data at hand, make an intelligent choice that termi*- 
nates his confusion." 

Polya (1954) states that the principle parts of a problem are the 
data^ ? the unknown, and the condition which links the unknown to the tfata. 
The* task of the problem solvei is then to (1) "understand" the problem, 
^•\i*.e., to specify the unknown, the data, and the condition, (2) to plan a. 
~ sQltiLion to the problem, i.e., to find the connection^ between the data 
'and the uVikfipwn , (3) to carry out the plan and (4) tb examine the solu- 
. tion, obtained .and/ determine if the* condition has been met. 

. A distinction made by Kramer further .clarifies ^the cdncept of problem. 
Kramer, spates that a given situation can be cither an enigma, a problem, 
or an, exercise-," dppendihg upon the "mathematical maturity" of the individual 
involved. ThuS^. \ one-step addition open number sentence (e.g<, ? + 3 = 9) 
is an enigma t;a thte threa-year-old who . cannot understand the question; 
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a problem to the six-year-old who understands the question but cannot 

automatically respond;and finally, an exercise to the eleven-year-old 

who can answer immediately. Gagne (1966) makes a similar distinction. 

He uses the term "productive problem solving" to refer to "... the 

finding of solutions to novel problems and should be carefully distinguished 

from an undesirable use of the term referring to routine substitution 

of numerical values in mathematical expressions of the same typp - a kind 

of "drill". 

The above statements give a general impression or the nature or / 
njfSblem salving tasks. The remainder of this sectidn further elucidates 
rfec^topic of problem solving in the context of the Word Problem ProgranN 
and specifies some of the overall objectives of the program. The thrSe 
subsections $eal with arithmetic word problem objectives, general problem 
solving goals, and the affective goals of the program. 



I . Arithmetic Word Problem Objectives 

The problems that will serve as our stimuli are verbal quantitative 
problems, i.e., problems which present the individual v^ith "a described 4 
situation that involves a quantitative question for which the individual 
has no ready answer" (Kramer ,^Jj9&#f\ The range of difficulty we intend 
to cover can be illustrated by the following objectives: 

- Given addition and subtraction word stories, thd student 

writes the number sentence. LIMIT: sums to 9 >1PI iftath Objective 
. 10-A) 

- Given a word problem for ^hich multiplication is the appropriate 
operation, the student writes a corresponding multiplication 
fact and, writes the answer with the appropriate label. LIMIT.: 
two one-digit factors (IPI Math Objective D - MULT - 2) 

- Given a word problem that requires addition, subtraction, or 
addition and subtraction, the student solves the problem and 
writes the answer with the appropriate label. LIMIT: four 
addends per problem, six places per problem, whole number parts to 
thousands, decimal fraction parts to millionths. 

(IPI Math Objective F - ADD/SUB s 4) 
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- Given a word problem that requires from 3 to 5 steps and the 
application of exactly two operations, the student solves the 
problem using a calculator and writes the answer with the 
appropriate label . 
This range will include and transcend the objectives referring to 
word problems currently in IPI Math. Behavioral>objectives and learning 
hierarchies which specify exactly what this entails are left until a 
later date.^-The success of the Program in attaining such objectives 
will be evaluated in the following manner: 

Objective f: We expect students utilizing the program to score 

significantly higher on Curriculum Embedded tests thart 
controls using the IPI approach. 
Objective 2: We expect students 'utilizing the program to score 

^ significantly higl er on Post tests than controls using 
the IPI approach. 
Objective 3: We expect students utilizing the program to score 

significantly higher on word problem sections of IPI 
placement tests administered in the succeeding year 
than controls usir.g the IPI approach. 
Objective 4: We expect students utilizing the program for two years 

to place out of significantly more word problem objectives 
than controls using the IPI approach. 
Objective 5: We expect students utilizing the program to score 

significantly higher on the word problem sections of 
standardized achievement tests administered at the 
end of the year. 

We also expect students to develop skills which will increase their 
competency in solving arithmetic and algebraic word problems of a more 
general nature. We strive to make it possible for. many arithmetic 
situations which are now f, enigm^" to become "real" problems to 
elementary school children. Thus, a. major goal of the Word Problem 
Program is to develop in the student effective problem solving strategies 
which we expect to transfer to other more complex word problem situations. 
As a measure of the success of the program in fulfilling this goal, students 
utilizing the prpgram are expected to score significantly higher on 
special tests -of "more advanced 1 ' word problems- than controls using the 
IPI approach. The term "mo^e advanced 1 ' refers^ to problems which are, 
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by some criteria, above the level of experience of the particular 
student involved and for which no instruction has been received, \For 
example, the tests may consist of problems requiring more step&^md/or 
operations, and a -greater degree of structural complexity than those of 
the highest level that the student has mastered. The students will al$o 
be given problems which contain numerical values of a greater Magnitude 
than they are accustomed and asked to specify the correct operations 
to be used. 



II . General Problem Solving Goals 

In addition to teaching competency in solving arithmetic word 
problems, the program is also intended 4 to develop skills in general 
problem solving, i.e,, to develop general skills and strategies whose 
application is not limited to the specific case of arithmetic word 
problems. Such general skills are hypothesized to exist by Olton (1969). 
"They are genera), cognitive abilities, such as the production of original 
ideas, the invent ioiTof a unifying principle which integrates several 
disparate events, and the use of various strategies when one is 'stuck 1 
on a complex problem." ; 

In order to specify the nature of the aforementioned skills , 
a more" precise definition of the term "problem" should be presented Newell 
and Simon (1972) state that M a person is confronted with a problem when 
he wants something and does not know immediately wj^at series of actions 
he can perform to get it." The desired "object" may be something tangible/ 
-concrete ox abstract, it may be specific or general, and it may be 
physical &r V)qnbolic. The series of actions he should perform can be 
physical, perceptual or purely fhental activities. In particular, the 
problems which concern us in the Word Problem Program have the following 

* characteristics : *> 

1. They are verbal representations of concrete, physical situations. 

2. The task may involve the re^resentat ion of concrete data in 
symbolic fashion . 

3. The actions required by the problem solver to attain the solution 
involve mental activities. 

4. No specific algorithm exists (applicable to the general problem 
solving situation) to produce the solution. Productive general 
strategies do exist, and the learner's ultimate task may be 



to find one. 
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5. There is only one solution to the problem, but several 
paths may exist to the solution. 

6. The problem may involve su-b-problems which must be solved 
to reach the final solution to the problem. 

7. The environment provides feedback as to the relative tenability 
of a given step in the solution process. 

8. The environment provides the opportunity to test the 
tenability of the final solution to the problem. 

The above list of characteristics represents our operational 
definition of a problem. The goals of the program are thus derivative 
from this definition. The goals are: 

GOAL 1: We expect students using the Word Problem Program to develop 

the skills nc^es^ary to solve problems as defined alcove. Some 
of these skills are listed here. These aretaken primarily 
from the work of Roman (personal communication) : 

1. Recognizing the existence of^a problem and clearly 
stating the goal . 

2. Recognizing, recording, and organizing the given data. 

3. Determining if analogous problems have been % solved and, 
if a general strategy or algorithm exists, recalling and 
applying it. 

4. Breaking an apparently complex problem into manageable sub- 
parts and solving each part. 

5. Reintegrating and substituting the sut>solution{ s) into a 
final solution for the problem. 

6. Creating hypotheses that are reasonable, consistent with all 
given data, and testable. 

7. Devising tests for the current hypothesis and integrating 
the evidence from those tests into the next hypothesis. 

< 8. Recognizing when the solution has been attained, or recognizi 
when help is needed, or recogrrf&zing when to giv^up. 
9. Communicating to another about the* problem and the attained 
solution. 

10. Extending the solution of a particular problem to a more 
general problem situation. 
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GOAL 2: We expect students using the program to develop important 

mathematical -concepts such as those of variable, transitivity, 
and substitution. Some specific examples of thi^ are given 
in sections 4 and 5. 
GOAL 3: We expect the program to serve as an introduction to algebraic 
concepts, reasoning, and manipulation of symbolic quantities. 
The evaluation of the success of such goals is difficult since 
few objective measures of such cognitive abilities are available. One 
group of tests which may prove helpful here are those developed by the 
Wisconsin Research" and Development Center for Cognitive Learning. A 
goal of their project was to identify and develop tests to measure 
basic concepts and cognitive abilities. We thus expect students 
using the Word Problem Program to score significantly higher on these 
tests than controls. 

III. Affective Goals - 

The specification and evaluation of affective goals presents an 

m 

even greater problem than that of the general problem solving objectives. 
Krathwohl et al (1956) define affective objectives as those which 
emphasize "a feeling tone, an emotion, or a degree of acceptance or 
reject ion. " Such objectives are generally expressed as "interests, 
attitudes, appreciations, values, and emotional sets or biases. " The 
statement of these objectives in specific behavioral terms is, however, 
exceedingly difficult. Subsequently, since tests for the attainment 
of objectives involve measurements of the presence of certain behaviors, 
evaluation is even more difficult. At present, it is not our intention 
to deal fully with thes« problems and thus the remainder of this "section 
is devoted to an overview of our current thinking. 

We are interested in the program users 1 behavior in three areas: 

1) toward the Word Problems Program, i .e. ^towards instruction via 
the program as opposed to textbook, booklet, or class, 

2) towards arithmetic word problems as compared to other subject 
areas, interests, and activities and, 

3) toward problem solving in general. 
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The affective goals of the program arc: ^ 
GOAL 1: In each of- the above areas, we expect? all program users to behave 

at Krathwohl's level of (1.2) willingness to receive, i.e., all 

users should be disposed towards (willing to tolerate and not 

avoid) the three areas. 
GOAL 2: We expect at least 2/3 of the usors\to attain behavior at some 

level of "responding" . This includes^ 

(2.1) Acquiescence in responding: The user is active in 
responding but passive in terms of the initiation of 
behavior to the three areas. 

(2.2) Willingness to respond; The user voluntarily chooses 
to engage in activities within the program and in 
problem solving. 

(2.3) Satisfaction in response: The user's behavior in the 
three areas is accompanied i>y "a feeling of satisfaction, 
kn emotional response, generally of pleasure, zest, and 
enjoyment. 11 ( 

GOAL 3: We expect a minority (approximately 1/4) of the users to attain 
behavior at some level of "valuing". This includes: 

(3.1) Acceptance of a value: The user is motivated towards 
activities in the three areas stimulated by a "belief" 
that these activities have some worth or value. 

(3.2) Preference for a vilue : The user not <only accepts the 
value of activities in the three areas, but actively 
pursues such act^ities. 

(3.3) Commitment: The u ^ er is motivated to action based on a 

"conviction" 4that the three areas have value. 

> 

There will be no directed effort to inculcate these attitudes, 
appreciations , and values. Rather, we efcpect the program itself to 
provide sufficient reinforcement for thqB attainment of these goals. 
Furthermore, these are primarily long r^gtp goals in that we do not 
expect many of these goals to be met "itnjaffl the first year. Instead, 
they are expected tJD be met at various PW|Us along a user's history. 
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Finally, evaluation of the success of these goals can consist 
of two types of activities: 

1) direct observation of user behavior by teacher and/or 
experimenter and 

2) user performance on specially devised tests (based oij 
Krathwohl's taxonomy) to measure the presence of these 
behaviors. 

Both of these metRods, however, have disadvantages. The first involves 
a great deal of time and %he possibility of lack of control and experimental 
bias. TheAecond involves the creation of a test whose validity would 
be subject to careful scrutiny. Thus, the decision regarding exactly 
which method(s) to utilize has not been made at this time and will be post- 
poned to a later date. 



7 



12H 

43 



2.01 



\ RESEARCH IN WORD PROBLEMS 

Research in arithmetic word problems has been conducted in a 
number of diverse settings and with a variety of different objectives. 
Studies have been made on: 1) affective variables and individual 
differences, 2) sstudent skills, abilities, and methods, 3) instructional 
techniques, strategies, and methods, and 4) structural variables hy- 
pothesized to affect word problem difficulty. This review will present 
a brief sample of the research which is relevant to developing a 
curriculum designed to teach qompetency in the ^olution of 
arithmetic word problems. 

A study by Gorman (1967) has proved useful* in the process of 
selecting pertinent research. Gorman collected and critically analyzed 
all available research on arithmetic word problems conducted from 1925 
to 1965. After the preliminary selection of 293 studies, he applied a 
set of arbitrary criteria and rejected 70% of the studies for failure 
to meet his requirements. The criteria stipulated that the studies 
must : 

1) pertain to written problems in elementary school mathematics 

2) include pupils in some of grades K-6 in the population 

3) be conducted between 1925 and 1965 

4) report results based on valid and reliable instruments and 
procedures 

5) be available from interlibrary loan, manuscript procurement, 
microfilm purchase, etc. 

6) be the original study (whenever possible) 

The remaining studies were then examined for compliance to criteria of 
internal and external validity, i.e., in order that a study be accepted, 
it must provide control for every factor which influences internal or 
external validity. The eight fa6tors (based on Campbell and Stanley, j 
1963) which affect internal validity, are : history, maturation, testing 
instrumentation, regression, selection, mortality, and the interaction 
of selection and maturation, history, testing, orfonortality . The 
external validity was evaluated according to the interaction of testing 
and the experimental variable, selection biases and the experimental 
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variable, reactive effects of the experimental variable, and' multiple 
treatment effects. After^this final analysis, 37 studies were accepted 

The Gorman study was u$cd as a tool in selecting and evaluating 
relevant research. Unless otherwise noted, the studies reported here 
from*! 925 -65 are those accepted by Gorman. The studies are listed by 
topic. 

I. Student Skills 

Balow (1964) investigated the relationship of reading and compu- 
tational ability to problem solving ability. As measures of these 
abilities, he used three subtds^ of Stanford Achievement Tests: 
Word Meaning and Paragraph Meaning, Computation, and Reasoning. The 
study was performed on 468 children chosen; from a group of 1400 sixth 
graders. Balow found that: £ 

1) Both general reading ability and computation ability have an effect 
on problem solving ability. Controlling I.Q. drastically reduces 
the strength of the Relationship. 

2) For a given level of computation ability, problem solving ability 
increases, as reading ability increases, and far a given level of 
reading ability, problem solving increases as computation ability 
increases . 

Gorman sites four other studies which examine such relationships 
and summarizes them as follows: 

1) Factors most closely associated with success in problem solving are 
those related to numbers and reasoning while factors least closely 
related pertain to vocabulary and reading abilities. (Hansen, 1943) 

2) AbilUly in fundamental operations and tests of problem reading (or 
problem analysis) have a higher correlation yiftth tests of problem 
solving than do tests of general reading, flStfcvens , 1932) 

3) Even though intelligence and computationalJability are factor^that 
cause differences in problem solving performance, other variables 
are also evident. However, reading ability as measured by achieve- 
ment tests is not an important factor in problem solving ability. 
(Engelhart, 1932) 

4) A complex interaction prevails between reading comprehension, 
reasoning, process selection, computation, and problem solving. 
(Martin, 1963) 
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Riedesel also reviewed the literature on elementary school 
education in Mathematics. On this basis, he identified eight factors 
which arc associated with high achievement in problem solving. The 
factors were: intelligence, computational ability, ability to estimate 
answers, ability to analyze problems, arithmetic vocabulary, ability y 
to use quantitative relationships that are social in nature, ability to 
note irrelevant (superfluous) detail, and knowledge of arithmetic con- 
cepts. He sites seven research works in support of this conclusion. 



II. Instructional Methods / 

Research on instructional techniques and methods of teaching 
competency in solving word problems has traditionally consisted of 
making comparisons of the effectiveness o"f ^:wo or three methods. 
It seems, however, that every researcher has a different concept or 
operating definition of the exact nature of those methods. For example, 
the term "Formal Analysis Method" has been used to refer to a three, 
four, five, or six step method of analysis. Since the authors are not 
always explicit in their definition of terms, one is forced to infer the 
treatment used in order to attain any useable and comprehensible 
comparison. In this section, research is grouped according to method. 

The methods shall be defined as follows: 

1) Practice-Only. This method appears under many different names in 
the literature primarily as a control method. It is, however, rarely 
defined explicitly and several variations can be seen to exist. The 
definition that seems most applicable is that which Morton (1925) calls 
the "Individual Method" in which the students are presented problems 
and left "to their own devices" (in HorSman, 1940). 

2) Vocabulary Methods are methods which attempt to improve the student's 
Jollity to solve word problems by instructing him in (usually spec- 
ialized) vocabulary. 

3) Norton defines the Analogy Method as follows: 

The pupil is given an easy oral problem which is similar to a 
difficult written problem. It is presumed that the pupil can solve 
the easy oral problem, see the analogy tp> the difficult written problem 
and then be able to solve the latter. 
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4) Morton (1925) defines the Dependencies Method (or the Method of 
Graphical Analysis) as follows: 

The pupil is taught to diagram the procedure to be followed. He 
begins r by determining what is to be found, then observes that this 
depends upon the data that are given in the problem, that these data 
depend upon othefs, and so on until the diagram has been completed. 
This diagram is intended to assist the pupil in outlining his thought 
procedures. It emphasizes relationships directed toward the solution. 

5) The Formal Analysis Method 

Several variations of this method are also in existence. For 

example, Morton defines it as a three step analysis of the problem, 

r 

whereas Washburne and Osborne (1925) consider it a six step process 
which includes a step directing the student to estimate th^ magnitude 
of the result. All definitions, however, include the following steps 
as crucial to the analysis: (1) Determine what is to be found, i.e., 
specify the unknown, (2) determine what information and numerical data 
arc given in the problem, and (3) decide which processes (operations) 
are necessary to attain the solution. 

6) Other Analysis Methods 

The sixth classification includes analytical or ^quasi-analytical 
techniques other than that of Forma] Analysis. 

The results of the research on instructional methods are summarized 
according to each of the aforementioned methods. Under each method, 
the research is divided into categories on the basis of the methods 1 
superiority, inferiority, or lack df significant differences to other 
instructional methods. Thus, the reader can quickly refer to any 
method and gain insight into its comparison to other methods. This, 
of course, necessitates some duplication under each method. 

1) Practice-Only 

i0 The Practice-Only method was found to be superior to the follow- 
ing methods: 

- the Formal Analysis method in a study by Hanna (1929, reported 

in Horsman)! Hanria called Practice-Only the "Individual Method 1 '. 

- (Formal) Analysis by Washburne and Osborne in 1926 (not reviewed 
by Gorman). In this study, the Practice-Only method was referred 
to as the "Many Problem" method and was considered to be "decidely 
the most effective method of all." 
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- the Analogy method by Washburne and Oj^ome, 

- the •'meaning" or "insight" method for students of relatively 
low intellectual ability and high achievement in a study by 
Anderson (1949, reported in Gorman)'. In this study, Practice- 
Only is called the "Drill Method". Gorman does not define "the 
meaning method. " 

ii) The Practice-Only method was found to be inferior to the fol- 
lowing methods : 

- the method of providing "systematic instruction" in the funda- 
mental processes in a study by Pace (1959) reported in Gorman. 
Systematic instruction was defined as a process in which 
students "are asked to explain how a problem is to be solved 
and why a particular process is appropriate". 

- the Wanted-Given method (see Other Analysis methods) by Wilson 
(1964) reported in hpth Gorman and Jerman (1971). 

- the Vocabulary Method by Vanderlinde (1962, in Gorman). It 
is presumed that the term "control method" refers to Practice- 
Only. 

- the "meaning" method for pupils of high intelljectual ability 
and low achievement by Anderson (above). x 

iii) No significant differences were found between the Practice- 
Only method and the following methods: 

- the Dependencies method by Hanna 

- a technique which encouraged students to estimate their 
answer before seeding the solution in a study by Dickey (1934 ,\ 
in Gorman) . / 

- the Action Sequence method (see Other Analysis methods) by Wilson 

- a modified Wanted-Given program (see Other Analysis methods) by 
Jerman (1971). It is again presumed on the basis of the report 
that the "non-treatment" groups engaged in Practice-Only. 

- The Productive Thinking Program (see Other Analysis methods) 
by Jerman. 

3) Vocabulary Methods 

i) A vocabulary method was found superior to the control method 
(presumably Practice-Only) by VanderLinde. VanderLinde used the "direct 
study* 1 technique of studying quantitative vocabulary which enabled the 
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child to ''establish a three-way association between the written symbol, 
the sound of the term, and at least one of its meanings/ 1 He found that 
students who had used this technique achieved significantly higher scores 
on a test of "arithmetic problem solving and concepts." 

ii) a vocabulary method was found inferior to an Analogy ftethod by 
Theile (1939, reported in Gorman). The vocabulary method involved the 
completion of mathematical problems with the correct term. 

3) Analogy Methtods 

i) Analogy methods were found superior to the following methods: 

- a step-by-step (Formal) analysis method by Theile. Theile used 
"the association method, or that technique by which difficult 
or incorrect problems are associated with a model." 

- a Vocabulary method by Thielc 

ii) The Analogy jnethod was found to be inferior to the following 
methods : 

- (Formal) analysis for the lower half of the children in a 
study by Washburne and Osborne. They state that Formal Analysis 
is "decidedly superior." 

- the "Many Problem" technique (Practice-Only) by Washburne 
and Osborne. 

iii) No significant differences were found between the Analogy 
method and Formal Analysis for the superior half of the children in a 
study by Washburne and Osborne. They state that "training in the seeing 
of analogies appears to be equal or slightly superior to training in 
formal analysis for the superior half of children." 

4) The Dependencies Method 

i) The Dependencies Method was found to be superior to the following 
methods: 

- the "Conventional-Formulae" (Formal Analysis) method by Hanna. 

- the "Individual Method" (Practice-Only) for the lower third of 
the students used by Hanna. 

iii) No significant differences were found between the Dependencies 
method and the following methods: 

- Practice-Only by Hanna 

- Formal Analysis in a study by Horsman (1940) 
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5) Formal Analysis 

i) The Formal Analysis methcyi was found to be decidedly superior 
to the Analogy method by Washburne and Osborne when only the "lower 
half" of the subjects were considered. 

ii) The Formal Analysis method was found to be inferior to the 
following methods: 

- the "Individual Method 11 (Practice-Only) by Hanna 

- the M Many Problem 11 (Practice -Only) method by Wd^hburne and 
Osborne 

- the Association (Analogy) method by Theile 

- the Dependencies method byTlanna 

iii) No significant differences were found between the Formal 
Analysis method and the following methods: 

the Analogy method when only the superior half of the subjects 
are considered by Washburne and Osborne 

- the Dependencies method by llorsman 

With regard to the literature on the Formal Analysis method, 
Suydara and Riedesel (1969) report evidence from five other studies 
which support thei? conclusion that "informal procedures are superior 
to following rigid steps .... If this analysis method is used, it 
is rscoriccr.ded that only one or two of the steps be tried with any 
one problem . " 

6) Other Methods of Analysts 

Wilson (1964, in both Gorman and Jerman) compared the effective- 
ness of three instructional procedures: the Wanted-Given , Action 
Sequence, and Practice On>£ methods. The Wanted-Given program: 

focuses on the goals and "tools", the "why" and "with 



what" . the ends and 



means, the wanted and givens in 
situations from whi(h the meaning or "attributes" of^n 
operation is to be abstracted. Thfs program emphasizes 
"why" and "with whali" one adds, subtracts, etc. Hence, 
the operations are jfconceived of in terms of their charac- 
teristic ends-means. Or, in other words, the operations 
* are relationships, or have structures; the rational 
attributes of which are wanted-givens. 
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In the Wanted-Given approach, the subjects are instructed to: 
, (1) recognize the wanted-given structure olf the problem 

(2) express this structure as an equation 

(3) compute by using the operation indicated by the equation. 
As an example, the problem 

Bob had 9 marbles. Dick gave him 3 marbles. How 

many marbles did Bob have then? 
was, classified as "a problem in which the size of a total is wanted H 
and the sizes of its parts are given" and the Wanted-given structure 
of addition is recognized. ^ The child must then solve the appropriate 
equation (9 + 3 = n or n = 9 + 3), The problems used in the study 
were all one step, one operation word problems and the equations 
referred to in (2) were to be of the "direct" type, i.e., with the 
unknown isolated on one side of the equation. 

The Action-Sequence method is described (in Gorman) as a 
program 

- focu$ina on wh^t is going on, what events went 
on, what is being done, what is done, what was 

the sequence of events, etc., in situations from 
which the meaning or "attributes 11 of an operation , 
is to be abstracted. Thistyrogram emphasizes 
what one does mentally or physically v^hen one is 
adding^ subtracting, etc. Hence, the operations 
are conceived of in terms of their characteristic 
act ion-sequente . In other words, the operations 
are relationships, or have structures, the rela- 
tional attributes of which are action-sequences. 

In the Act ion -Sequence approach, the directions were: 

(1) 'see' or recognize the real or imagined action-sequence 
structure of a problem. * * 

(2) express the action-sequence in an equation. 

(3) compute, using the operation indicated if the equation 
is direct; if the equation is indirect, imagine an 
appropriate second action sequence; express it as an 
equation and compute using the operation indicated. 

(4) check by rewriting the equation with the answer in the 
proper position. 
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Wilson found that the Wanted-Given program produces statistically 
significant improvement in verbal problem solving ability over both 
the Practice-Only and the Action-Sequence methods, lie failed to 
attain significant differences in favor of Action-Sequence over 
Practice Only. 

Jerman (1971) performed a study which compared the effectiveness 
of The Productive Thinkfpp Program , a Modified Wanted-Given program 
and a control group. The Productive Thinking Program 'is a commercially 
available jgreries of programmed comic books in which children learn to 
solve problems by following the exploits of a brother-sister-uncle trio. 
The program is designed to promote the generalized problem-solving 
skills of elementary school students. Jerman summarizes the program's 
purpose as follows: "to develop and strengthen a student's ability in 
using important skills and strategies for thinking and problem solving, 
to improve a student's awareness of his own thinking processes, and to 
improve his attitude toward activities that involve, use of the mind." 

The Modified Wanttfd-Given program developed by Jerman differed 
from Wilson's program in that it asked the student to progress through 
a number of, steps (dependent on the number of, steps in the problem) 
by making a choice between two alternatives at each step, l^e prob- 
lems were identified as either sum- or product -type problems as 
opposed to the wanted-given whole-part relationship of the Wilson 
program. Throughout the program, emphasis was placed on the meaning 
of the terms "sum", "addends" etc. and the number of rules to be learned 
by th6 student was kept to a minimum. Finally, the students were 
allowed (even encouraged) to write indirect number sentences where 
appropriate and to use these to generate the direct equation and sub^^_ 
sequently attain the solution. The results of the study were as follows 

(1) No significant difference was found between scores of students 
using The Productive Thinking Program and the Modified Wanted- 
Given program. In a follow-up study Seven weeks later, significant 
results favored the Modified Wanted-given. 

(2) No significant difference was found between the treatment and 
control groups. 

(3) A significant difference (p <..001) was found between treatment 
and control groups in the number of correct procedures used by 
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students to solve the problems. Also, t;he Modified Wanted-Given 
favored (p < ,005) The Productive Thinking Program in this respect. 



Ill, Structural Variables 

Structural variables are inner- or inter-problem variables whose 
presence, absent, or magnitude is hypothesized to affect the difficulty 
of a particular word problem. The most recent studies regarding 
structural^ variables have been conducted by Loftus and others at 
the Institute for Mathematical Studies in the Social Sciences 
at Stanford University, These have been conducted within the context 
of a Computer Assisted Instruction Program for sixth grade students, 
T!fe program was unique in several ways. First, it did not require^ 
the students to perform the actual arithmetic computations, but 
did compel them to indicate the operations and operands in an unusual 
fashion. It required from four to eight weeks for the students to 
master the instruction set. Furthermore, the results may not be 
completely generalizable sinca the experiment was performed on what 
Loftus (1970) refers to as "slow learners". Since the implications 
of the Stanford studies are directly relevant ta the present work, 
a brief description of the format of the Loftus word problem 
program will be given here. 

The program typed a word problem on the printout sheet and 

followed it with a printout, of the numerical data which appeared 

in the problem on separate, numbered lines, A typical problem from 

the instruction set follows: 

27 CHILDREN GOT 13 PIECES OF CANDY EACH, 

GENEROUS GEORGIA GAVE AWAY 9 PIECES, 

HOW MANY PIECES DID SHE HAVE REMAINING.,, , 

G (1) 27 

G (2) 13 

G ,(3) 9 

The "G" indicates that the numbers were given in the problem. In 
order for the student to perform an operation, he had to specify the 
line (not the number) and the operatiop in suffix notation. Thus, 
to solve the above problem, the student would type 1,2 M which meant 
"multiply the number on line (1) by the number on line (2), or 27 X \V\ 
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The results were computed by the prograifT and displayed with the ^ 
next^onsequtive line number. To indicate that his answer could be 
found on a certain line, the student typed 5X^ yeaning "the number 
on line (5) is my answer". The student could also' enter a new number 
\into the machine (for conversion problems) and be given hints 
(during the instruction phase), e.g., for the problem "What is (486 ♦ 
390) ♦ 707?", the hint is "First find 486 ♦ 390. Then add that sum 
to 707". The format of the Stanford CAI program, and its effect upon 
the experimental results, will be discussed in more detail elsewhere. 

The results of the Stanford^and other studies are presented 
here according to^topic. 

1. The Operations variable is defined by Loftus as the miniftum 
number of different operations required to reach the correct solution. 
Thus, a problem requiring a student to add seven numbers necessitates 
one operation, whereas a problem requiring the average of three 
numbers requires two operations. The operation variable can assume 
only the values 1, 2, 3, or 4. The hypothesis was that the larger 
the number of operations required, the more difficult the problem. 
This variable was found to be significant in (Loftus, 1970; Suppes 
Loftus, and Jerman, 1969; and Loftus and Suppes, 1972)* 

2* The sequential variable involves the relationships between 
problems presented in a sequence. The hypothesis was $nat a problem 
is easier if it can be solvedjyy the same operations in the same order 
as the prepeeding problem. This variable was found significant by 
Loftus; Suppes, Loftus, and Jerman; and Loftus and Suppes. The 
"arrangement of problems within a series" (sequential) variable was 
also fbund significant by Hydle and Clapp (1926, a study rejected by 
Gorman) . 

3. The depth variable, as defined by Loftus, is a measure of the 
structural complexity yf the sentences within a word problem. Based 
an a procedure defined by Yngve (1964), a number is assigned to each 
word in a sentence depending on how "imbedded" the word is in the 
sentence's structure tree diagram. The mean of these ^umbers (com- 
puted for every word in the sentence) is taken and the measure of 
"depth" becomes the greatest numerical value of the means of all 
the sentences within a problem. This variable was found to be 
significant in studies by Loftus; and Loftus and Suppes. 
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4. The length ■variable was defined by Loftus as the numbe r of 
words in the problem. This was significant in Loftus; and 
Loftus and Suppes; but not in Suppes, Loftus and Jerman. . 

5. The conversion variable was found significant in studies 
by Loftus; \Suppes, Loftus, and Jerman; and Loftus and Suppes. 
The hypothesis was that^5rofcl>£ms requiring conversion of units are 
more difficult than those which do not. 

6. The verbal clue variable was not found to affect problem 
difficulty significantly by Loftus;' Suppes, Loftus, and Jernan; 
and Loftus and Suppes. The hypothesis was that problemis which 
contained a verbal clue to the required operation were easier than 
those which did not. The verbal clues were defined as "and" for 
addition, "left" for subtraction, "each" for multiplication, and 
"each" or "average" for division. 

7. The steps variable was defined by Loftus as the minimum number 
of steps required to reach the solution. Each application of any 
operation is considered a step. Thus , ^ pr obi eii requiring a student 
to add seven numbers requires six steps (only one operation) whereas, 
a problem requiring a student to find the average of three numbers 
requires three steps (and two operations). This was insignificant 

in studies by. fcoftus; Suppes, Loftus, and Jerman; and Loftus and 
Suppes. 

8. The order of numerical data variable refers to the hypothesis 
that a problem is easier if the numerical data within it are presented 
in an order in which they can be used to solve the problem. This was 
found to be true in a study by Burns and Yonally (1964), and was found 
insignificant in studies by Loftus; and Loftus and Suppes. 

9. The order of fundamental processes variable was investigated 
extensively at the University of Pittsburgh by Young and others. 
Young (1941) had hypothesized t;hat the order in which the fundamental 
arithmetic processes Coperations) occur in problems would affect the 
difficulty of the problems. To test this hypothesis both Becker (1938), 
and Berglund^Gray (1938) generated a set of two-step, two-operation 
problems. The set of problems was divided into six subsets based 
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on the syc possible combinations of four things (the arithmetic 
operations) u&£d two at a time (additionvsjubtractio^rablens , 
addition-multiVlication, etc.). The uraer in which^ic processed 
appeared in the problems was then rcvers64. to generate a second | 
with all other variables remaining constant. For example, ^/a 
in the first set would be solved, by first applying addition 
then subtraction, whereas the analogous problem in the second set 
Would be solved by subtracting first and then adding. Berglund-Gray 
found the difficulty Qf interpreting arithmetic problems is 
definitely affected by the order of occurrence of the fundamental 
solution process" and summarizes in the following table: 

t 

More Difficult Less Difficult 




Subtraction - Addition ^ Addition - Subtraction 
Addition Multiplication f Multiplication - Addition 
Division - Addition Addition - Division 

Division - Subtraction Subtraction - Division 

Subtraction - Multiplication Multiplication - Subtraction 

Division - Multiplication Multiplication - Division 

where "Division - Multiplication" refers to the problem in which 
division is performed first anp multiplication second. Becker had 
similar findings but did not include all of the cases. Mc Isaac 
(1940) performed the same experiment on three-step, two- or three- 
operation problems and found the order of occurrence equally significant. 

An important flaw in the above, studies is that many of the problems 
can be solved in an order othe^r* thah that which the authors assume 
they will be solved. Also, the process used to generate "identical" 
problems With a reversed occurrence of operations does not, in reality, * 
control for all other variables. The problems of the reversed 
sequence are often far more complexly worded than the original. This 
has the effect of increasing the structural complexity af the 
problem (the ''depth" variable). It should be noted that ^either 

Yngve noj; Transformational grammar were available in 1940. Gorman 
t 

rejected all four of these studies for failure to isolate the experimental 
variable. ■ , . 
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10, The sequence of information of arithmetic word problems was 
investigated by Rosenthal (1971). Rosenthal generated 32 addition 
and subtraction word problems based on the following item forms: 
m ^ n s ? and ? ^ n = p. F.ach problem involved the subject 
"starting out 11 with "m" objects , gaining or losing "n" objects 
and ending up with "p ,f objects. Either "m" or "p" was chosen for ^ 
the unknown quantity. Three hypotheses were investigated: 

(1) There exist significant differences between problems generated 
from eac/^of the four item forms (which specify whether the "unknown 

is the starting or ending set). 

(2) That problems in which information was presented in the 

forward temporal order would be easier than the reverse. For example, 

for the M m ♦ n = ?" item form, the problem 

If Paul started out with 5 boats and he bought 
3 boats, how many boats did he end up with? 

in which the events are mentioned in their proper temporal sequence 

will be less difficult than 

How many boats did Paul end up with if he bought 3 
boats, and he started out with 5 boats? 

which is stated in reverse temporal sequence. 

(3) There will exist significant differences in problems due 
to differential effects associated with the problem verb. 

Significant differences were found for hypotheses (1) and (2), 
but not for (3) . 

llyThe position of the question was investigated by Williams 
and McCreight (1965). Their hypothesis was that placing the question 
at the beginning of the problem will result in significant differences 
when compared to placing the question at the en4 of the problem. They 
found that the problems in whfch the question appeared at the beginning 
of the problem were slightly ( p .1) easier. ) 

12. Other variables 

Ilydle and Clapp (1926) investigated some other variables in a 
massive study involving from 5870 to 7029 subjects. They found that 
problems with an abstract objective setting (problems containing 
Situations which are difficult to visualize) and those which contain 
unfamiliar objects, large numbers, symbols, or non-essential (super- 
fluous) information are significantly more difficult than problems 
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which do not contain these variables. Gorman rejected this study 
because of factors affecting internal validity. 
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INSTRUCTIONAL METHODS IN THE WORD PROBLEM PROGRAM 



In the Word Problem Program, instructional decisions have been 
made and are currently being made on the basis of mathematical and 
educational philosophy, educational psychology, the results of research, 
and on the basis of intijition. Some of these decisions are discussed 
in this section according to the following scheme: 

1. Thc& Sequence of Instructional Objectives: 

This subsection relates the rationale and method of ordering 
objectives within the program. 

2. Instructional Strategies: 

This subsection relates the other teaching strategies 
employed by the program to meet the objectives. 

3. The Role of Computation: ^ 
This subsection relates the rationale behinfl" an~I)istruct ional 
decision concerning the relationship of computational ability 
to problem solving ability. ' 

Later sections of the paper expand upon some topics which appear 
in this section. These deal with the program format and the relationship 
of word problems to number sentences. 



I. Sequence of Instructional Objectives 

In Goals for School Mathematics (1963) the authors state that "... 
problem material should be considered at least as important as the text 
proper... we therefore believe that the composition of problem sequences 
is one of the largest and one of the most urgent tasks in curricular 
development." In the* Word Problem Program, the textual material is. 
virtually eliminated and thus, the manner in which sets of word problems 
arc sequenced will bear the 'brunt of the instructional load. 
Th^ assumption is that a precise ordering of instructional objectives 
(sets of problems) from easy to difficult (simple to complex) will' 
prove to be an effective instructional procedure. 

The use of precise sequences of instructional objectives which form 

learning hierarchies has been suggested by Gagne(1968, 1970). Gagn& 

hypothesized that a given task which was to be learned could be analyzed 

into simpler prerequisite capabilities. Thus, by successively asking 

4 

the question "what would the individual already have tn know how t^ 
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in order to learn this new capability simply by being given verbal 
instruction? 11 , a list of subordinate capabilities may be identified. 
The 1 learning of these subordinate tasks should facilitate positive transfer 
of learning to the learning of* the superordihate task, i.e., the 
superordinate task "will be more readily learned (on the average, through- 
out a group of students) if the subordinate capabilities have been 
previously acquired and are readily available foryrecall . " 

£he implication of Gagnd's work for the Word Problem Program is 
that sets of problems should be individually analyzed and ordered 
according to the skills necessary for solution. However, the task 
of identifying subordinate, skills in word problems is more qomplex than 
i/i other areas where this analysis technique has been successfully 
applied. To verify this statement, one need only examine the research 
which has been conducted on structural variables in word problems. 'Such 
research represents an attempt to delineate factors which contribute 
to problem difficulty and are therefore thought of as analogous to the 
skills necessary to complete the problem. As evidenced in the review 
of the literature, even this simplification o£ p> Gagn& type task analysis 
is often complex and sometimes yields contradictory results. For this 
reason, the results of research on structural variables lias rarely 
been utilized directly in the design of curriculum. One of the first 
to use this method with some degree of sqccess was Gill (1940). 

"Gill compared the effectiveness of the method of providing fouirth 
and fifth grade students with problems of "graded difficulty" to a f 
control method (Pyactice-Only) . He generated sets of 7 or 8 problems 
which were given to the students in 26 lessons over two 13 week semesters. 
Gill ordered the problems by directly applying the results of existing 
research on structural variables to the task of writing the problems. 
The variables which had been found to affect item difficulty we^e 
individually arranged from easy to difficult along the entire continuum 
of 26 lessons, i.e., each particular variable was graded on a Scale and 
applied by order of difficulty to the problems in the lesson. All of 
these variables were then incorporated to write the graded set of 
problems. The variables which were taken into consideration were: 
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1. The order of the fundamental processes. 

2. The size of numbers. 

♦ 3. The objectivity (the ease with which a problem can be 
M visualized M ) . 

4. The ocpHTence of unfamiliar objects in the problem. 

5. The arrangement of problems (the Loftus sequential variable). 

6. The occurrence of nonessential or superfluous information. 

7. The use of symbols in the problem. 

Gill found superior achievement in the experimental group on standardized 
arithmetic tests, a test of word problem solving which did not require 
arithmetic computations, and a test designed to samplA the structural 
variables at different levels. Gill's study was rejected by Gorman for 
failure to control the teacher variable (the same teacher did not teach 
both groups except in two of the 16 classes). 

On the basis of the aforementioned thSbry and of Gill's research, 
it seems reasonable to assume that a hierarchical ordering of sets of 
word problems will be an effective instructional strategy. Once the 
pedagogical decision to hierarchically sequence the word problem objectives 
has been made, the problem of how to successfully perform this task 
becomes paramount. This could be accomplished in several ways. It could 
be done empirically by testing large numbers of children on each problem 
to be used and subsequently performing the necessary statistical analysis. 
This method, however, woyld prove impractical due to the immense numbers 
of problems which would be involved. A reasonable alternative would be 
-an application of Gill's method based on current research. 

In reviewing the literature on structural variables, the Loftus 
study stands out as being particularly applicable to such an undertaking. 
In her study, Loftus determined regression coefficients for the eight 
variables she examined (five \of which she found significant). On the 
basis of the coefficients, a difficulty index can be determined for a 
given problem by identifying the value of each variable in the problem 
and plugging these values into the regression equation. A total set of 
problems could thus be ordered on the basis of the difficulty indices 
computed for each individual problem. 
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Unfortunately, it does not appca* that the Loftus results will be 
directly applicable to the ordering of problems in the Word Problem 
Program. This is due to several factors in the study which cast doubts 



on- its validity and generalizability . The first such factor is the 
awkward method in which students were expected to respond to the problems. 
* The format, which is described briefly in the review section, >^s so 



complex that if^requi red from four to eight weeks for the students to 



•master the instruction set before beginning the (four week) problem set. 
Secondly, the population consisted of "slow learners" and, hence, may 
not be completely generalizable. 

With regard to the regression equation itself, Loftus states that 
two particular problems contributed most heavily to the tot*l chi square 
and that "the regression models investigated cannot account for perfor- 
mance on these two types of problems." In addition, the results ' 
concerning two of the eight variables studied can be contradicted 
by other research. Finally, since other researchers have identified 
other significant variables, it is clear that the Loftus variables do 
not take all of the relevant factors into consideratioflj. The nature of 
the regression equation, precludes its modification to include new 
variables v*i%hfAit a massive amount of work. Consequently, the Loftus 
results should not be applied as the sole basis for organizing and 
^S£qtfencing the problems in the Word Problem Program. 

Several possible approaches to ordering a set of word problems 
have been considered. As stated previously, the method of treating 
each of several hundred problems separately and investigating student 
performance was rejected as impractical. Gill's work was rejected as 
out of date and the Loftus study was performed in an inappropriate 
population of students and does not include all significant variables. 
The classification of all problems on the basis of structural variables 
is still desirable, however, and the problem thus becomes one of deciding 
which variables to consider and exactly how to use them. A crucial 
point to remember is that research on structural variables represents 
an attempt to deal with the problem solving skills which are required 
in a particular problem. Thus, the depth variable is presumably signi- 
ficant because of additional skills W concepts which are required 
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of the learner for problems of greater syntactic complexity. It 
is therefor/ desirable to choose variables for consideration which are 
directly related to (hypothesized) problem solving skills. Clearly, 
variables cannot be ignored if they have consistently proven to be 
significant in previous research. Other variables must b6 ignored, either 
because they have no close relationship to skills necessary to solve 
the problenror because previous research has shown them to be insigni- 
ficant. JFor these reasons, a new variable (or set of variables) is unjier 
consideration for use as a major tool for organizing the Word Problem 
Program objectives. This will be discussed in detail in the section 
on Number Sentences and Word Problems. 

II . Instructional Strategies 

The choice of which instructional strategies to employ in the Word 
Problem Program should be made on the basis of both the objectives of" 
the program and the results of research in the area. The objectives t>f 
the program are basically aimed at two goals: The development of skills 
to increase competency in solving arithmetic word problems and the 
development of more general problem solving skills. Ideally, previous 
research should give some indication, as to which methods will be 
effective in attaining these goals'. 

As evidenced in the reviey of literature, there is much debate 
and controversy over the question of whether to teach specific word 
problem solving strategies or algoritluns (and if so, which ones), to 
teach general problem solving skills, reading skills, or simply to 
employ the Practice-Only Method. Research has yielded contradictory 
results as to which methods are more effective. For example, the vocab- 
ulary method was found superior to Practice-Only by VanderLinde. Since 
Practice-Only was found superior to both the Analogy and Analysis 
methods, then one should conclude that the study of vocabulary would 
also prove superior to Analogy and Analysis. This was not found to be 
the case, however, since Thiele, using a different technique, found the 
Analogy method more effective than the study of vocabulary. Thus, 
care must be taken in attempting to apply the results of research 
directly to program development. 
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Research can, however, prove helpful in the decision as to which 
methods to employ in the Word Problem Program. Intuitively, it seems 
logical that to accomplish the program's goals, the program should make 
use of an Analysis or Analogy method. Presumably, utilization of one 
or both of these methods would aid in developing problem solving 
strategies and skills applicable to specific word problems as well as 
general problem solving situations. Research, however, indicates that 
the Formal analysis method would not prove effective in teaching 
competency in arithmetic word problems. As mentioned earlier, Suydam 
and Riedesel conclude that ''informal procedures are superior to following 
rigid steps... if this analysis method is used, it is recommended that 
only , one or two of the steps be tried with any one problem." This 
conclusion, along with the favorable reports on the effectiveness of 
"Other Analysis Methods" which utilize number sentences (Wilson, Jerman) , 
support the method described below for the Word Problem Program. 4 

In the Word Problem Program, a structure of hints available to 
the student will be used to foster an analytical apptfff&ch to problem solving*. 
The use of this approach will be encouraged but not demanded and is thus 
made available at the student's option. When a student is experiencing 
difficulty with a particular problem, he may type "HINT" and a low 
level hint will be supplied. Subsequent use of the HINT command will 
provide additional hints until the final level is reached. The levels 
of hints are as follows: 

0. If the problem is one ^requiring a conversion of units, tjie 
appropriate conversion will be made available. 

1. The first level hint will be one borrowed from Polya. In 
Polya's approach to problem solving, the first step is to 
identify the unknown. The first hint on the word problem 
program will thus be a specification of what the student 
is expected to find. 

2. The second hint will consist of a rewording of the problem 
in simpler (shorter) terms. The rationale here is to reduce 
the problem to a simpler problem by removing all of the less 
relevant information. In this sense, it is similar to an 
analogy method. 

3. The final hint consists of a translation of the word problem 
into a corresponding number sentence. Tfre number sentence will 
not necessarily have the unkrown isolated on one side of the 
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equation, i.e., it may be indirect. 1 
The intent of the hint structure is therefore to encourage the 
student to take the following steps in problem analysis: 
1. identify the unknown ^ 
- r 2. translate the word problem into simpliet terms 

3. translate the new version into an arithmetic number sentence 
(or a collection of such sentences) and 

4. solve the number sentence (s) 

This approach, though analytical in nature, is not a Formal Analysis 
method. It resembles Formal Analysis in that a step-by-step procedure 
is encouraged, but it also resembles a vocabulary approach (the 
translation of the initial word problem), and the analogy methqd (the 
student solves the simpler word problem). It is most similar to the 
experimental programs of Wilson and Jerm^p^see Other Analysis Methods 
in the review) in that both of these recognized the role of the number 
sentence in the solution of word problems. In the Action Sequence and 
the Wanted-Given programs, the number sentence was seen as a natural 
representation of the events in any ^iven word problem. In this program, 
it is seen as a tool, i.e., as an abstraction of the problem situation 
in a step towards solution. The exact nature of the relationship 
of the number sentences to the Word Problem Program will be elaborated 
upon in the next section. > 

Finally, the manner in which problems are presented to the student 
and the form in which the student must respond (the program format) is 
expected to help attain the program objectives and is thus part of the 
instructional strategy of the program. This will also be discussed in 
the last section. . * 



III. The Role of Computation 

The numeric values in word problems play both a confounding and an 
undeniably important role. For example, Suydam and Riedesel state that 
studies "...reveal that pupils often gi\se little attention to the actual * 
problems; instead they almost randomly manipulate numbers. " We expect 
k that this is not far from the truth, but tend to believe that such V 
manipulation of numbers is more algorithmic than random. For example, 
Stevenson (1925) relates a method of word problem solvi^i'^ed by an 
* elementary school student as follows: >^ 
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If there are lots of numbers, I add. If there 
are only two numbers with lots of parts, I subtract. 
But if there are just two numbers with and one 8 
littler than the other, it is hard. I divide if 
they come out even, but if they don't, I multiply. 

Conclusions made on the basis of an analysis of number sentences in the 
IPI Functions program indicate that similar algorithms are used by 
children working on number sentences. This will bek discussed in detail 
later along with some suggestions for dealing with the situation. 

The research on the sequence variable also seems to support the 
above arguments. Both Loftus;and Hydle and Clapp feund that a given 
problem is easier if it can be solved by the same operations (in the 
same order) as the problem that preceded it. This could also be 
interpreted as an indication that students respond to a given problem 
with an algorithm which first instructs them to apply, or attempt to 
apply, the solution strategy from the previous problem, Regrettably, 
no research has been performed to determine whether a significant 
number of errors on word problems arc made when a student (mis)applies 
the operations he used in a previous problem, e.g., perhaps a student 
will be more likely to incorrectly multiply in an addition problem if 
it is presented in sequence after other multiplication problems. If 
such a situation exists, the solution is obviously to present sequences 
of problems in a mixed fashion (i.e., take the sequence variable into 
account) in an attempt to dissuade, rather than reinforce , the 
attainment of such algorithms. 

The above arguments serve the purpose of depicting the confounding 

nature of the numbers in word problems. The major topic of interest 

here, however, is the correlation between computational skills and problem 

solving ability documented in the review section. While several 

researchers have found a^positive correlation, the work of Jerman 

Qualifies th^s. As mentioned previously, Jerman noted no significant 
differences between the treatment and control groups in terms of numbers of 
correct responses but did report highly significant differences in favor of 
the experimental group in choosing the correct procedures. This finding 
Indicates that the process skills and the computational skills may be 
considered independent and therefore can be treated as such in the 
curriculum. Thus* the development of computational ability can be under- 
taken separately from the development of problem solving skills. This 
is exactly the intent of the Word Problem Program. 

/ ' • 
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The separation of computation from processes been attempted in 
the past by researchers concerned with structural variables. In 
n particular, Bcrglund-Gray and others at the University of Pittsburgh, 
performed their research under conditions in which the subjects were 
not required to perform arithmetic calculations. Instead the answer 
sheet consisted of a 4 X 2 matrix (for two step problems) with columns 
labeled "addition", "subtraction", etc. and rows numbered "1" and "2". 
To solve the problem, the student had to indicate the process by placing 
an "X" in the first row under the operation he would apply first and 
another "X" in the second row under the operation he wo/ld apply next. 
This method allowed the researchers to investigate problem solving without 
contending with the computational variable. The method did, however, 
have drawbacks. Since the subjects indicated only the operations they 
would use, the researchers had no assurance that the subjects were 
indicating the proper operands., Furthermore, although the researchers 
attempted to use only unambiguous problems, it was still possible to 
solve a given problem correctly using two different sequences of 

operations. The researchers, however, accepted only one of these. r 

More recently, Loftus used the computer to separate the computa- 
tional and problem solving components. In the Loftus program, the 
subjects specified the operands and the operation to be used to solve 
a particular problem, and the machine did the computation. The manner 
in which the program format forced the subjects to reply (discussed 
elsewhere) was, however, unsatisfactory. 

In the Word Problem Program, the computer will allow us to perform ^/ 
all calculations for the student withdut the negative aspects previously 
mentioned. A detailed description of the manner in which problems are 
presented and in which the student is expected to^respond can be found 
in the section on Program Format. 



ERLC 



67 

153 



4.01 



NUMBER SENTENCES AND WORD PROBLEMS 

» 

Several researchers and educators presume a relationship between 
word problems and number sentences. This relationship is generally 
manifested in the role which number sentences play as a step towards the 
solution of word problems.' In the Word Problem Program, we utilize 
number sentences in this fashion, but also hypothesize a different 
sort of relationship which involves the use of number sentences as a 
structural variable to aid in sequencing word problems. This section 
describes these and other uses of number sentences in the Word Problem 
Program. 



to \ 



I. Number Sentences as Instructional Strategy 

Riedesel, on] the basis of his review, urges teachers 

make of mathematical sentences in solving v 
single and multi-step problems. This approach \ 
f improves pupil ability to look for the salient * 
aspects of problems and then select symbols that ex- 
* press this sense. Thus pupils can be taught to 
grasp the structure of the problem before "looking 
for the answer". However, care should be taken 
not to use the mathematical ^sentence as the only 
way to solve problems. 

The Seeing Through Arithmetic (STA) program by Ilartung et al., 
(1967), incorporates this strategy to the fullest extent. The STA 
program teaches two methods of problem solving. Students in the 
second and third grade are taught by the Action Sequence Method 
described in the review (Wilson based his method on the 1,964 STA approach.) 
In this approach, students were to recognize the real or imagined 
action-sequence structure of a problem, express it as a number sentence, 
and solve the nun^ei^scntence. However, students could only solve 
direct number sentences and thus, for some problems , they had to 
"imagine an appropriatef^second action sequence" and express it as 
a (direct) number sentence. * 



68 

ERiC 154 



4.02 

f 

In the fourth grade, the STA approach changes and the student 
is directed to follow three steps of a general problem solving proce- 
dure : 

(1) obtain from the verbal problem a mathematical sentence that 
describes the situation. 

(2) process the sentence to gel the unknown number in accordance 
with certain mathematical properties and definitions. 

(3) Interpret the' end result of the processing in terms of the 
original situation . 

The method takes the student from the more concrete physical 
situation of the Action-Sequence method to the "mathemat ical world 1 ' 
of number sentences. The student is encouraged £6 find a direct 
or indirect number sentence to describe the problem situation. If 
the equation is indirect, he may solve it by applying what he has 
learned about the commutative property, the relationships between the 
operations, and about ''related sentences." The method relies heavily 
upon the last concept. "Related sentences" are sentences which contains 
the same numbers and have the same solution, e.g. 3 X ? = 12, 12 ■ ? X 3, 
and ? » 12 / 3 are all related sentences. The student is expected 
to "know" the related sentences for any given sentence, and choose 
the one that tells him directly what operation to use. He then computes 
the solution and returns to the "physical world" to interpret it. ^ 
The authors of STA state that this "general procedure can be used to 
solve all kinds of problems, no matter how difficult the problem may 
be." 

The Wanted-Given programs of-Wilson and Jerman also make use of 
number sentences. Wilson assumed that there exists an "essential" 
relationship between "the "known and the unknown" or the "givens" and 
the "wanted" in word problems. The "wanted" number is seen as a particular 
kinci of number: "a sum, a difference (or unknown addend), a product, or 
a quotient (or unknown factor)' 1 . In the Wanted-Given program the student 
is taught to recognize the wanted-given structure of the problem and 
to express it as an equation. In accordance with the program's 
definition of the arithmetic operations and the wanted-given structures, 
all of the number sentences should be direct and should not require 
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further modification for solution. Wilson's program is only 1 used, 
however, with one step word problems. Finally, Jerman's Modified 
Wanted-Oiven program also relics heavily upon number sentehces . In 
Jerman's program, however, the s^jUcnts are encouraged to write indirect 
equations as partial solution to the word problems. 

Each of the ajpove in€,tructional njcthods involves the use of number 
sentences to teach competency in solving arithmetic word problems. The 
methods differ in the manner in which the s.tudent is required to /derive 
a number sentence from a given word problem. In the Word Problem program, 
the student is encouraged (though not required) to represent word 
problems as number sentences. This is accomplished through the HINT 
structure de^pribed in the preceeflim; section. The HINT structure 
suggests the strategy of first translating a given word 'problem into 
simpler terms* (a modification of the analogy method) and subsequently 
translating the simplified problem into' a (direct or indirect) number 
sentence. Thus, Ihe principle differences between The Word Problem 
program and the aforementioned programs with regard to number sentences 
are : 

(1) . utilization of *thc proposed problem solving strategy is 

optional in the Word Problem program and mandatory in the 
other programs. * 

(2) The Word Problem program encourages a strategy o£ translation 
rather thlan one of recognizing hidden ^'structures 1 ' and "relation- 
ships' 1 in word problem's. Wo believe that this translation 
strategy will prove to be applicable to a wider range of 
problems than will the other strategies. 

Since number sentences will play such an integral role in the Word 
Problem program, it is necessary to provide ^instruction in all aspects 



of their §olu>tion. Fpr this reason, a mincer sentence program will 
also be developed which will parallel t>he Word Problem program. JThus, 
a student will be given a particular word problem only if he has 
already mastered objectives on number sentences identical to those 
attained from the wtfrd problem. Finally, the significance of number 
sentences to the Word Problem Program is manifest in both the method 
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and the form in which the student is expected to respond to a given 

word problem. This will be discussed in the section on program format. ' 

II. Nurfber Sentences as a Structural Variable 

In the review of the literature, several structural variables were 
described. Only the study by Rosenthal, however, dealt directly with 
the relationship of word problems to number sentences. Rosenthal 
studied three Variables, one of which was concerned with number sentences. 
He generated word problems to correspond to number sentences of the 
form M N = ? and ? N = P. Rosenthal found that problems that 
correspond to the form M ^ N ■ ? were significantly (p < .001) easier 
than those which correspond to the form ? +_ N * V . This result 

indicates that tfrere is indeed a strong relationship between word 

t - . 

problems and nqmbej sentences. The purpose of this subsection is 

to relate the manner in which this relationship can be used to help 

answer some questions concerning word problems. 

Many of the structural variables discussed in the review can be 
seen to be directly analogous to conditions present in number sentences. 
The most obvious of these are the steps, operations, and position of 
question variables. Other such variables, however, may be inferred. 
For example, the order variable (of numerical data) would be present 
if the numerical data in the number sentence did not* correspond 
in order to that of the corresponding direct number sentence. In 
mult i -operation number sentences, the order of operations variable 
may have a similar effect on number sentences as it did on word problems. 
Finally, one can even hypothesize a relationship between the struc- 
tural complexity of a word problem and its corresponding number sentence. 

On the basis of Rosenthal's findings and the above analogies, it 
seems reasonable to assume that number sentences, due to their simplicity, 
can aid in hierarchically ordering word problems. The question which 
now arises is how to utilize the relntionship fruitfully. One obvious 
problem is that several different number sentences can be derived 
from a given word problem and, inversely, a multitude of word 
problems can be generated for any given number sentence. Two methods 
are under consideration -to deal with such problems and enable us to 
utilize number sentences in ordering sets of word problems. 

I 157 

71 



4.05 

In the Tirst method, wc momentarily assume that we can translate 
a given word problem into a unique number sentence. After translating 
a set of word problems to their corresponding number sentences, wc may 
either: 

(i) utilize the resultant number sentence as a structural 
variable along with the sequential, conversion, length, 
and other syntactic variables to sequence the word 
problems using a Loftus-typc regression equation, or 
(ii) order the word problems purely on the basis of the number 
sentences, i.e., group the problems according to number 
sentence form and order as if they were number sentences; 
finer ordering (within groups) can then be performed on 
the basis of other structural variables. V 

Both of these alternative approaches to the first method require 
the generation of a number sentence from the given word problem. To 
be at all meaningful, the generated number sentence must be unique. 
This can be accomplished through the use of an algorithm which con- 
sistently generates the same number sentence for a given word problem. 
An example of an algorithm with this capability is the STUDENT program 
developed by Bobrow (1964). Bobrow's computer program can solve 
algebra word problems which have been written using a specific subset 
ofthe English language. The program accomplishes this by a series 
of translations of the text of the word problem into equations, and a 
subsequent solution of the equations. The existence of the STUDENT 
program confirms the plausibility of creating an algorithm which can 
translate a word problem into a unique number sentence. 

The second method also involves an algorithm similar to the 
above, but makes use of the algorithm itself (as opposed to its 
product) to determine the difficulty of a problem. The algorithm 
would be more complex than that needed for the first method and thus, 
would probably be based more directly on Bobrow's work. The scheme 
for ordering the word problems is to generate an algorithm which 
solves . problems by processes and operations analogous to and representa- 
tive of those used by human subjects. An index could be generated on 
the basis of type and number of processes (or loops in the* program) 
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necessary to solve the problem/ Sets of problems could then be 
ordered on the basis of individual indices. 

The rationale behind the generation ai*J utilization of such a 
complex algorithm is that it will produce results more directly 
representative of item difficulty tHf^n utilization of the structural 
variables discussed previously. The algorithm should give us a method 
for comparison of problems on the basis of variables more directly 
analogous to the actual skills and processes necessary for a student 
to solve the problems. The Bobrow program stands as evidence that 
an algorithm can be produced to solve word problems, and research 
is currently under considerati >i* tfhich will give an indication of the 
feasibility of applying such an algorithm to the task of ordering 
sets of problems. This research will be conducted using the analogous 
case of number sentences only, i.e., generating an algorithm to solve 
number sentences, or4ering on the basis of an index computed by the 
algorithm, and checking the hypothesized ordering against empirical 
data . 

The choice of which method to employ will be made primarily on 
the basis of time, feasibility, and resources. The order of preference 
for the alternatives is: method 2, method 1-i, and method 1-ii. 
Regrettably, this is also the order of difficulty. In the prclimirary 
version of the Word Problem Program', a simplification of met h ^ l^ff % 
was used as a method of stuping sets of similar problems. This method 
was based on the~ direct number sentence and thus required no algorithm 
to generate unique number sentences for each word problem. Although 
this method seemed quite feasible and elaborations could be made with 
relative simplicity, it lacks the exactness and refinement of the more 
difficult and time consuming techniques. 



Ill . Other Uses of Number Sentences 

It has been indicated several times that the study of number sen- 
tences could aid in the study of wor 1 problems. An example of this is 
a recent, informal study of student irotocolS on objectives in one-step 
number sentences taken from the IP1 Junctions Program. Correct and 
incorrect responses were recorded nnd the incorrect responses were 
classified as resulting from errors in either computation or process. 
An error was classified as a process error only if the value attained 

ERIC 73 l r > 9 



4.07 



by the student could l>c exactly calculated by ^application of the inverse 
operation on the two operands. Problems were/ also classified as belonging 
to one of six types (three when assuming symmetry about the equality 

relation) : ' ./ 

I 

Type 1: ?=aob, aob=?/ 

Type 2:a = ?ob,?ob = a/ 

Type 3: a r b o ?, b o ? * L 
where "o" symbolizes one^of^he four operations . The results of a chi 
square test yielded significant diff/erences between types (X^ = 57.48, p < 
.001) and further analysis showed the following: 

(1) Symmetry about the equality relation. Within each type, no 
significant difference wis found between problems in which 
the operation appeared on the left or the right hand side of 

• the equation. This finding allowed us to consider three 
rather than six types. 
In an attempt to account ^or the differences between types, the total 
errors were classified as either due"*t^process (8% of the total) or 
calculation (8% of the total). 

(2) No significant differences were found between types of problems 
on the basis of errors in calculation 

(3) significant differences were found between types (X2 =54.32, p < .001) 
on the basis of errors in 'choosing the correct process. 

In light of these results, a further analysis of individual 
problems allows us to conclude that many of the students were operating 
under a peculiar and undesirable algorithm which relies heavily upon 
the operation symbol which appears in the problem and the particular 
nature of the numeric values in the problem. As an illustration of this, 
in the problem 6 / 3 = ?, the M / M cues the operation of division and 
the divisibility of the numbers supports this. However, the problems / 
6 / ? ■ 3 and ? / 3 * 6 are solved in the same fashion, the first 
correctly and the second incorrectly. In the second problem, where 
multiplication is required to produce the solution, the M / H and the 
divisibility of 6 by 3 cues the operation of division. Note, however, 
that whereas ? / 3 ■ 6 cues cjivision, ? / 17 = 5 does not, and thus, 
due to the lack of divisibility, the correct (inverse) operation of 
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multiplication will be chosen by default. By the application of such 
an algorithm, a surprisingly high percentage of problems can be answered ; 
correctly without the slightest understanding or application of 'the 
mathematical principles involved. An analogous situation exists 
in addition and subtraction problems. 

The above situation can be seen to correspond to the phenomenon 
described by Stevenson and discussed in the proceeding section. This 
supports the suggestion that many children solve word problems by an . 
undesirable algorithm, i.e., individual words or phrases cue operations, 
the validity of which can be checked on the basis of the divisibility 
or subtractibility of the numbers in the problem. If this is indecxl 
the method employed by student; to solve word problems, then efforts 

4 

should be made to discourage its use. Two methods are under consideration 
to deal with this problem. 

The first method prescribes that the student progress through 
the word problem and number sentence curricula in three levels: 

(1) The first level allows only "legal" numbers as values in 
the problem. That is, no numbers of a magnitude or nature 
(e.g., decimals or fraction;) to which the student has not 
been exposed will appoar in the problem or in the solution. 
This is the "real" level in which the aforementioned 
undesirable algorithm works. 

(2) The second level is designed specifically to mislead on^ who 
blindly applies an algorithm like that described above. For 
example, in the number sentence program, the numbers can be 
chosen randomly (to allow negative numbers and decimals as 
answers), or can be of grca' cr magnitude than that to which 
the student has been exposed. Lack of familiarity with 
the numbers should not impede performance since all actual 
calculations will be performed by the machine. Also, numbers 
t;hat arc easily divisible will be used in problems requiring 
the student to multiply the numbers. The rationale for all 
of this is to eliminate the cues that the numbers themselves 
provide and, thus, to force the student to consider other 
reasons for choosing a particular operation. In the case 
of word problems, care must be taken to generate problems 
whose numbers do not cue operations and yet do not result 
in physically absurd problem situations. 
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(3) The third level will consist of problems in which only symbols 
appear. This purely abstract case is used to insure the 
understanding of the processes by the child. 

The second alternative method constitutes a reversal of the first. 
The proposal is to teach the third level first, so that students begin 
by solving number sentence problems and, subsequently, word problems, 
with abstract symbols. The rationale here is to avoid the formulation 
of the algorithms mentioned. This would do away with the difficult 
task of leading the student to forget his old algorithm and relearn 
another. Furthermore, we believe that teaching the manipulation of 
abstract symbols first will develop skills which will transfer to 
the case of real numbers. This stands, however, in contradiction to 
the method of proceeding from the concrete to the abstract case and 
thus, the question of feasibility is paramount here. Clearly, this 
method should be neither accepted nor rejected without further study. 
A major portion of the proposed research on number sentences will 
be devoted to determining the feasibility of having elementary school 
students work with abstract symbols. 
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FORMAT OF THE WORD PROBLEM PROGRAM 



As stated previously, the manner and form in which problems are 
presented and in which the user must respond will make an important 
contribution te the oveYall instructional effectiveness of the program. 
Several formats have fyeen considered for use in the Word Problem 
Program and have been tried in an experimental context designed to 
simulate on-line computer conditions. One of these has been chosen 
for implementation. This section presents a discussion of t;he proposed 
format and a description of some alternative formats. 

The proposed format was derived principally from that used in the 
Loftus study described in the review. It represents an attempt to 
refine the Loftus format into one which is far simpler and more 
appealing mathematically. The program proceeds as follows: 

(1) A word problem is typed on the screen. 

(2) The numeric data presented on the problem is listed beneath 
the problem (in order of appearance in the problem) and 
labeled with consecutive alphabetic characters, each on a 
separate line. 

* (3) The user must then respond with either "HINT" to get a hint, 

"DONE" to quit, "INSERT" to enter a new number fori a conversion 
problem, the option descried in (5), or, he may perform 
some arithmetic operation. To do this, he types the operands 
(represented by alphabetic symbols) separated by an operation 
symbol . 

(4) The computer performs any calculations indicated by the user 
and displays the value labeled with the next available alpha- 
betic character. The user is again given the options defined 
in (3). 

(5) To indicate that his answer has been represented by an alphabetic 
character, the user types "ANSWER" followed by that character. 
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_ As an illustration, the following is an example of the interaction 
between the program and a student solving a one-step word problem. The 
student's responses follow the "***" and arc underlined: 

HOW MANY COINS DID BOB COLLECT 

IF MARK HAS 10 COINS LESS THAN BOB 

AND MARK COLLECTED 20 COINS? 





GIVEN 


A . 


10 




GIVEN 


B 


20 


*** B-A 


B-A 


C 


10 


*** A+B 


A+B 


D 


30% 



- *** ANSWER D 

Two points of interest immediately arise: that of the format's 
practicality and its pedagogical value. With regard to practicality, 
preliminary studies indicate that students master the commands of the 
program in less than an hour, whereas the Loftus format required from 
4 to 8 weeks (Loftus does not say how many hours) of practice 
problems to prepare the students for the word problem set. In terms 
of the pedagogical value, we hypothesize that jnany interesting and 
important mathematical skills .ind concepts can be shaped through the 
use of this format. We expect that utilization of the format will, 
by itself, accomplish many of the program's objectives in the realm 
of problem solving abilities, For example, to solve the above sample 
problem, the student must be able to associate the symbols P 10 M and 
"20" with the symbols "A" and "B". He must be able to represent the 
sum of the two numbers as f, A+B M and issociate the sum with the new 
symbol M D ,r . Finally, he must specify his answer by its single 
symbolic representation. The significance of this last association 
should be clearer in light of the following discussion. 

In one of the experimental tryouts of another format, a fifth 
grade subject was presented with a two-step multiplication word problem' 
and asked to find a single number sentence to correspond to the problem. 
He was told that he could ask the experimenter to write anything he 
pleased to aid his memory and that he would have to state this request 
and his "final solution. 
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The problem was: 

* IN ONE YEAR, BOB ENCOUNTERED 13 GROUPS OF MEN 
AT EACH BALL GAME HE TRAVELED TO. IF THERE WERE 
15 MEN IN EACH GROUP, AND .BOB TRAVELED TO 5 BALL GAMES, 
HOW MANY MEN' DID HE SEE, TOTAL? 



After a great deal of effort, the subject began to approach the solution 



S: . . .X equals 13 times 5 * 

000, so far so good 
X equals 13 times 5 
. X equals 13 times 5, so, X times 15 equals how many he 
saw in whole year, but I can't see how to £ut that; into 
3 sentence . putting that in a sentence. 
» X times, 

X equals 13 times 5 is, \) 
X plus, 

X times 13 equals X 
E: X equals (?) 

S: X equals 13 times 5, and X times 15 equals X 
E: (writing) X times 15 equals X 
S: a different X 
Era different "X M ? 
S: yeah 

E: What do you want- to call, this M X lf ? 
S: I'll call that X 
(pause) 

S: question mark " * 

E ; , (writes "X x 15 = ?") 't 

The subject was then reminded that he must express his answer* as 
a single nuftber sentence. After several attempts: 

S: I wonder how you caa make that in one sentence? 
13 times 5 equals X, 
thats my X right there, so 
- it must be 
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X times 15 equals question mark? 

no, cause you don't know how you got the other X 

to equa\ your question mark, so 

X equals 13 times 5^ 

13 times 5 equals X 

X times 15 equals question mark. T just can't figure how 
to do it . 

The subject finally gave up his search for a solution. He had 
no difficulty later, however, in solving a two-step addition problem 
which he initially attacked from a different direction: 

S: I'm gonna get into the same thing I got into the other one and 

then I don't want to do that. 
0 

Um, 25 plus 35 plus 19 equals 
Is that called one sentence? 
Ilis performance on this and other tasks indicate that his difficulty 
was not a lack of ability to generate two-step number sentences, but 
rather a lack of such fundamental processes as association and substitu- 
tion. Other subjects working in this format and iri the proposed 
format were found to exhibit similar behaviors. For example, in the 
proposed format, the same problem discussed above would be solved as 

follows: 5 

GIVEN A 13 

GIVEN B 15 

GIVEN C 5 

*** A^X B A X B D 195 

*** D X C 1) X C E 975 

*** ANSWER E 

Children initially had difficulty using this format to solve 
multi-step problems. On the basis of the trials with subjects, however, 
it is felt that training in responding to one-step problems in this for- 
mat will enable the student to perform successfully on multi-step problems 
That is, we expect that the process of associating and specifying the 
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computed answer in terms of a single abstract symbol (in one-step problems) 
will transfer to the tasks of association and substitution in the afore- 
mentioned two-step problems. In addition to these skills, the primary 
purposes of utilizing this format are to develop a mathematical concept 
of variable and to develop skills in abstract symbol manipulation 
which will hopefully transfer to other areas of mathematics and, especially, 
to algebra. 

Xs stated earlier, the above format has been chosen for implementation. 
Various other formats have been considered and are still being studied 
for possible use in either a complementary role or as a total replacement. 
The remainder of this section briefly examines some of these possibilities. 

Several variations of the proposed format are worthy of consideration. 
One variation is to allow the student to type the numbers themselves, 
instead of typing their symbolic (alphabetic) representations, e.g., 
"5 + 2" instead of "A + B". Another is to allow inputs containing more 
than one operation at a time, e.g., to allow "A ♦ B - C" (or "20 ♦ 
5 - 19") as an intermediate or final step. This variation, however, 
has undesirable aspects, such as the need for parenthesis to specify 
the order in which operations should be performed. These 'Variations 
have been tentatively rejected due to the hypothesized superiority 
of the proposed format for accomplishing the problem giving objectives 
listed in the first section. 

Another set of variations is based upon the experimental format 
described earlier in this section. In this format, the student is 
given a word problem and is asked to type a number sentence from which 
the solution tor the word problem can be derived. The computer will 
then solve the number sentence and display the solution. If the number 
sentence typed by the student is indirect, the computer will display 
the corresponding direct number sentence before the solution. The 
student, will then be asked if he thinks his solution is correct. If 
he r^ponds negatively, fee is asked to type a new number sentence. If 
he answers affirmatively, he is informed as to the correctness of his 
responses. This format is more directly related tfc the specific 
strategy whose use is promoted* in the HINT structures In effect, it 
assumes that* given word problem is considered solved when an appropriate 
number sentence is generated. Such a program must, of course, be 
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closely integrated with the Number Sentence Program to assure the 
veracity of this assumption. 

A modification of this format could allow students to solve a 
given word problem by writing more than one number sentence. The 
student could define variables in several sentences and combine these 
to form his answer. For example, a word problem with solution ? - A - B 
♦ C could be solved with the sentences X ■ A - B and ? ■ X ♦ C. As is 
the case of the proposed format, each of these formats could be 
implemented with either numbers or with alphabetic symbols which 
correspond to numbers. 

All of the above formats have been tentatively rejectefr*on the 
basis of experimental tryouts for feasibility and the format's 
hypothesized relationship to the program goals. They are, however, 
still under consideration for use in some manner. For example, it may prove 
beneficial to program more than one format and utilize them at f\ 
different stages within the curriculum. It is hoped that additional 
study will aid in the ultimate choice of program format. ' 
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EXPERIMENTAL VERSION O F THE WORD PROBLEM PROGRAM 

In the winter of 1972, an experimental version of the Word 
Problem Program was created. In this version, the program generates 
and displays word problems on demand in the format described in 
section 5. The purpose of this section is to describe some technical 
aspects of the experimental version and to presents some sample protocols 
of the program in operation. 

The existing program types the specified word problem and allows 
the user to perform calculations, insert his own numbers, or ask for 
hints. The facilities to present hierarchical sequences of problems 
and to judge which level of problems to present next have not yet been 
programmed. The program is written in a high level interpretive 
language which is extremely well suited to program development , but 
unfit for use in the school environment because of its slow operating 
speed. The program is thus being rewritten in a high speed compiled 
language for school implementation. 

The program generates word problems using a technique of item 
form generation. Certain words in the problem are left variable, i.e., 
at generation time, they may be chosen randomly from lists of appropriate 
words. Pot example, the NAMES list contains as elements the names: 
Lucy, Sue, Barbara, etc., and the BAG l^st contains: bag, pile, 
collection, container, etc. By using open grammatical sentence 
structures, many different problems can be generated from one item 
form. In the examples of item forms that follow, the variable elements 
of the form are written with the names of lists contained in paren- 
theses. This notation means that a word is to be selected at random 
from the named list. When suffixes follow the name of the list, 
they refer to different elements: (NAME, 1) means Jto select one 
element from the NAME list, and (NAME, 2) means to select another 
element from the NAKffi list that is different from the first chosen. 
If the same number appears twice in the item form, it refers to the 
same element already chosen, and not to a new random element. Two 
part lists also exist in which the first and second parts of the list 
are related in some fashion, e.£., singular and plural of the same 
word. Thus, (GROUPS, PI) refers to a randomly chosen element of the 
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first part of the GROUPS list, and (GROUPS, P2) refers to the 
corresponding second element. (PRONOUN, 1, NOM) refers to the 
nominative form of the pronoun for (NAME, 1). Thus, the symbolic 
form (NAME) (OBTAIN, P2) (A) (OBJECTS) could be translated as JOHN 
HAD 18 BALLS or as MARY FOUND 4 APPLES. The following are examples 
of item forms from two of the existing 14 classes: 

(1) Class MM (Multi-Multiplication) 

EACH OF' (A) (GROUPS, P2) (GAVE) (NAME) (B) (BAGS) 

OF (OBJECTS). HOW MANY (OBJECTS) DID (NAME) 

RECEIVE IF EACH OF THE (BAGS) CONTAINED (C) (OBJECTS)? 

(2) Class A/M (Addition/Multiplication) 

(NAME, 1) (OBTAIN, P2) (A) (OBJECTS). IF (NAME, 2) 
(OBTAIN, P2) (B) TIMES AS MANY (OBJECTS) AS (NAME, 1), 
HOW MANY (OBJECTS) DID THEY (OBTAIN, PI) (ADDWORD)? 
At present, approximately 125 item forms have be^n programmed, 
each with the capability of generating thousands of unique word 
problems. These item forms are divided into 14 classes on the basis 
of the operations and number of steps necessary for solution. In 
the following sample protocol, the first problem has been generated 
S from item form (1) above and the next two were loth generated from 
item for (2). As before, the Students responses are underlined. 

r 

EACH OF 7 MEN SOLD BETTY 12 BOXES 

OF CANDY BARS. HOW MANY CANDY BARS DID BETTY ^ 

RECEIVE IF EACH OF THE BOXES CONTAINED 10 CANDY BARS? 



*** AXB 
*** DXC 

*** ANSWER E 



GIVEN 


A 


7 


GIVEN 


B 


12 


GIVEN 


C 


10 


AXB 


D 


84 - 


DXC 


E 


840 


ANSWER E 




GREAT I 



ANOTHER? 
**♦ YES 
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BILL FOUND 23 STAMPS. IF JANE 

FOUND 6 TIMES AS MANY STAMPS AS BILL, 

HOW MANY STAMPS DID THEY FIND TOGETHER? 

GIVEN A 23 

GIVEN B 6 

*** HINT 

HINT: THE PROBLEM ASKS YOU TO FIND THE NUMBER OF 

STAMPS THAT BILL AND JANE HAVE, TOGETHER. 

*** AXB A ♦ B C 29 

*** ANSWER C ANSWER C WRONG ANSWER 

*** HINT 

HINT: THE PROBLEM IS SIMILAR TO THE FOLLOWING ONE 

IF JANE HAS 6 TIMES BILL'S 23 STAMPS, 

THEN BILL'S STAMPS PLUS JANE'S STAMPS = ? 

*** AXB A X B D 138 

*** A ♦ D A ♦ D E 161 

*** ANS E ANSWER E GREAT! 

ANOTHER? 

*** YES 

) 

JUDY BOUGHT 9 PEARS. IF SALLY 

BOUGHT IS TIMES AS MANY PEARS AS JUUY, 

HOW MANY PEARS DID THEY BUY IN ALL? 

GIVEN A 9 . 

GIVEN B 15 ' 
*** INSERT 135 INSERT C 135 

*** A ♦ C A ♦ C D 144 

*** ANS D ANSWER D GREAT! 

ANOTHER? 
*** NO 
GOODBYE NOW 
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Sister Claire McCormick's curriculum project, 
developed for this course and the introduc- 
tion and Conclusion of her Analysis of the 
NRS Reading Program completed during her 
internship. The detailed level-by-le^el 
analysis is lengthy and is omitted from 
this report. 
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project ur.de rtakc i a.; preparation for a.n in- 
ternship in the Reading rVo^raii. - l There are many paths 
of approach to a design: tec. terr^i 1.1 1 noi:it. The terminal point, 
in this case, is entry into a cu?-riculur. writing or curriculum 
development project. T'r.er:. are . t.v ways of going about ar. 
instructional tssk, of faci li'cat:. .ig the progress of the student 
fror: where he is to where he wan* 5 to be. The strategy chosen 
here is that of modeling. 

The model ir. this instance : - a small,, segment of the materials 
of level- 3 of the '\w Reading :> tc:.:. 7rorn level 3 have been 
selected four representati* e scj; *nccs written as cassette- 
scripts: 3-l-A/3-l-^, 3---H, ar i xhe observation of 
the model becomes a scrutiny whir *. seeks to extract from the 
materials the programming technic aes evidenced therein. 

The analysis of the pvtorir.Ic, then, speaks to the problem: 

* To what exter.t can scientific principles as developed 
in behavioral psychology and applied in programmed 
instruction be uti' ized in designing primary 
, grades reading matt rials for individualized 

instruction? 

It is intended that the res- Itir.g analysis provide an 
introduction to work in the tota! project. This should enable 
the student to enter with a bett* r preparation for the work of 
the project because the ac ,iviti< s here exemplified have pro- 
vided the needed familiarisation. 

* * * >*c - « * ^ * >" ^ ********* 

M • ' 

The content here presented V.cludes three, sequences on \ 
'the digraph ch_ and one sequence 1 he word families derived 
from ot, tt t and it. The coropor. nt skills in elude: response 

\ 
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difiV:-or.ti:it:-.on, r.r.uoci.atinn, (• i -crisunitio'.i, chianinc 

ar.d concept xornation. 

The concepts include thos<, r jc:-.c.-:v Ligations: 

• tinted -letters (symbois) represent spoken sounds and 

mean the same tfiing as the sound;: . 
Printed vmrds aro composed of sounds and sounds are re- 

oresentod by -printed symbols . 

There are root words which c~.n* have^endings attached. 

— There are word families which differ only in their initial 

phonemes • 

Since the program has beep, ery carefully developed, it 
can simply be stated -that the pr re^sitos for entry into the 
sequences" under study is success. 'ul completion of the prior 
sequences. For entry into the p . o-ram there are no require- 
ments specifically related So .'c .dir...;. The child who is ready 
for first grade in an American c.-ty school should be ready 
for this program. 



•/hat follows are four separate analyses of the four cas- 
sette-scripts mentioned above. 
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is the best of the "lot. 




'X can sing," said Tot , 
"and a fish can not'/ 




Copyrlr.hr 1971 rw i n/»rpir r»„. 



- UKiDCCL.Ki^C 0 ,r F f\fcU u y WORTHS 

MOCRL - KiiPUiN-T.KS • ULLV Tu LI Co fc*«TT Wofc&S 
f° ft » • i *s L - T>: Uy 'A ; c ^ 



fir i \\ FoViCH ■ A £mT . t ... , . . . _ i 

l , ^- ^ \ * > _ Co N ? 1 v; % AKTi > NJ 

184 * 

98 



BEST COPY AVAILABLE 



V v. 



i V/ 



:M "TV.- F'/J-.T 



To €r I \j £ 



Pert". 



TV v. ^f, '.\'. _7. - -v. 



Or |^l3.^VotiC£ 4 



Ft: ?■»■;. z-:. 



v- .... i . : . i • 



A 



(-.'_>0 UTILIZE 



IS 



ERIC 



185 

99 



CI 



2. 



I « • %m 



set 

let 
net 



pet 
Chet 



-47 



This is Chet. 
Chet is a pet 
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The man is on the " T.V. set. 
The man ic singing to nis pet. 
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The elf nam^d Nrt 
is sitting in a pit. 
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"That's the lamp I lit.' 
Said the HttSpr elf :>!!t. 
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Cassette Script y-'h-A 

Response Pa^es lyj, zsxi 1^5 



Lot's chock to sc-e that you — j or. the ri^ht pajo. You should see 
s-k-A in the box at the top cf your pa;e. (2 SECCttCD PAUSE) If you see 
in the box, jty. are on the ri^ht page and we can begin. (1 SECOND 

PABSS) 

Find the box next to numeral I. (1 SECOND PAUSE ) Point to the 
letters in box 1 as I say the sounc these letters make together. Ready: 
/ot/. JNow, you say the sound the letters make. (2 SECOND PAUSE) The 
letters o-t make the sound /ot/. 1 oint to, the letters and say their sound 
out loud with xe. Ready: /ot/. 

Next to numeral 2 there is a I ox vith five words in it. Do you see 
the box? (i SECOND PAUSE ) I'll rc^d the words in the box, you point to 
/each vord as I read. Ready by the first word; cot , not, pot , lot , Tot . 
Do you see what is the same about all those words? (1 SECOND PAUSE) All 
those words ei*l the same. They all end vith the letters o-t. The letters 
o-t make the sound /ot/. I'll read the words in box 2 again, but this time 

I want you to read each word outloui with ma. Ready: cot, , ■ not , not , lot , 

i 

tot . Now, I want you to read each word in box 2 yourself ,/ then draw a ring 

/ 

around the letters o-t, the lettfrri that make the sound /ot/. (STOP) 

Look at t the picture of the cat beside numeral 3. (1 SECOND PAUSE) Tot 
is the name of the cat pictured no>.t to numeral 3- Look at Tot. (1 SECOND 
PAUSE) Tot likes his naue so nrach he has it printed on his shirt. Read the 

9 " 

name on the cat's shirt with me. Foaiy: Tot . Tot not only likes his name, 

he likes all /ot/ words. Tot's favorite words ere all of the /ot/ words in 
box 2. 

Co to numeral 4. (1 SECOND PAUSE) There's a two line rhyrce next to 

mrusral 4. ^Point to each word as I read the rhyme. Ready by the first word: 
The cat /^v^cj ]y/_ — r V Jst of; J ^j_ ^ ou CCSi sco ^ ro:T1 the picture 
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that Tot doos think ho 1 a th~ bjst. I 1 11 road tho rhyme next to numeral ^ 
again, but vhon wo co:x- to an ur^Lr! ~ncd word, you r^ad that word outloua. 
Ready:' The cat named (2 SECOt?/) PAU3 :) is the best of the (2 SECOND PAUSE) . 
Did you road Tot and lot outloud? 

Go to numeral 5. (1 SBQjjlD ?AJ3E) tjext to numeral 5 is another two 
line rhyme. Follow along by pointin ; to each: word as I redd tho rhyme* 
Ready; n I can sing , " said Tot , " and a fish can not. " The picture shows a 
fish watching Tot sing. Hake an I o 1 the picture of the fish. (2 SECCIJD 
PAUSE) I f ll read the rhyme nuxt to numeral 5 again, but when we come toy 
an underlined word you ba sure to re.xl that v;ord outloud. Ready: "I can 
sin^, ■ said (2 SECOND ?AUSE) . " and a fish can (3 SECOND PAUSE) , n I hope you 
read Tot and not outloud. • 



" Turn to the next page. (5 SECO .D PAUSE) You should be on page one 
hundred thirty-four. (2 SECOND PAUS 0 If you ' are on page one hundred thirty- 
four f you/ are on the right page and ve can continue the lesson. (1 SECOND 
PAUSE) 

Find numeral 1 at the top of yo'.x paper. (1 SECOND PAUSE) Point to the 
letters in the box noxt to numeral 1 as I chy the sound these letters make. 
Ready; /ct/. Now, you say tho sour^. (2 SECOND PAUSE) Tho letters c-t make 
the sound /ct/. You say /et/ vith ttj . again. Ready: /ct/. 

Go to box 2. (y SECO:^ PAUSE) All of tho words in box 2 end with the 
letters e-t. The letters that say A t/. I ! ll read all of the /et/ words in 
_ . box 2. You follow along by pointing to each word as I read. Ready by tha top- 
word: met , set, l^t, rrt, p?t *,. Chot . This ^ime, I want you to read each word 
in box 2 outloud with ma. After \ Jed a word, I uill give you time to draw 
a ring around the letters £-t t the L Iter 8 that say /et/ in each word. Ready 
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by the top word in box 2: r.-l , . .Is. , ur~w u rii.j avoua-.l tho two letters 
that make the /et/ sound in iuut. (;, SEwGlCD PAUSE) Go to the next word. 
Heady to read: set. Ring the letters that mako the /et/ sound in set. 
(3 SECOJD PAUSE) . . next word, heady: let . . . ring the letters that 
make the /et/ sound in let .. (3 SECOND PAUSE) . . . next word . . . net . . . 
ring the letters that say /et/. (3 SECOND PAUS12) . . . next word . . pet . . 
ring the letters that say /et/ in jxt. (3 SECOND PAUSE ) . . . last word ... 
Chet , . . ring the letters that say /et/ in Chet . (3 SECOKD PAUSE) ' 

* 

Look at the picture of the monkey next to numeral 3» (1 SECOND PAUSE) 
Chct is the name of the monkey picti red next to numeral 3t Chet likes his 
namo so much he wears it on h^shi^-t. Do you cee Chet r s name on his shirt?. 
(1 SECOND PAUSE) 

Next to, numeral Us a rhyme that tells us about Chet. (1 SECOND PAUSE ) 
Follow along as I x-ead tho rhymo next to 'numoral k. Ready: This is Chet . 
Chet is a pet. Look at tho picture- (1- SECOND PAUSE ) The picture shows 
Chet walking down the street ^n.th a man. Chet is the man's pet. I'll read 
the rhyme next to numeral 2> again, '..hen we come to an underlined word, you 
read' that word outloud. Ready by n.-meral <\ This is (2 SECOND PAUSE) . Chet 
is a (2 SECOND PAUSE) . 

Go to mjmcral (1 SECOND PA.SE) Follow along as I read. Ready: The 
ran is on the T.V. sot . The nan is sin^in^ to his pet. The picture " shows the 
man singing to his pot. Make an X ..n Chct the pe*. [J, SECOND PAUSE) 

Turn to the next page. (5 SEC j$D PAUSE) Check to see that you sve on 
page ono hundred thirty-five. (2 S..'COKI)( PAUSE) We can continue the lesson 
if you are on page ono hundred thir y-five. (1 SEC^h) ...USE) 

Find numeral 1 at the top of y. ur page. (1 SECOND PAUSE) I'll say tho 
sound the letters in box 1 make. L/>ten: /it/. Now, you say the sound theso 
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letters make. (2 Sl^CCM) VfXZZ) :v.; It^er: j-t rr.ako the courxi /it/. S^y 
the ,souisi with me again. Ready: /it/. 

Go. to box;2. — (1 SECOND ?AUS2) I'll read the words in box 2, you follow 
along by pointing to each word as I read. Ready by the top word: sit , bit , 
git , pit , lit t ' Kit, All thoso wo?dJ end the sarae, they eniwith the /it/ sound. 
The /it/ sound is i&ade by thfc letters i-t * You> caake a ring around the letters 
in each word in box 2 that make the /it/ sound. , (STOP) Let's read the words 
in box 2 outloud together. 'Ready by the first word: sit, bit, fit, pit , lit , 
Nit. Now, read the last word in th.s box outloud by ; yourself . (2 SECOND 
PAUSE) lit If/ the last word. Nit is the name of the elf pictured next to 
nurceral 3. Look at Nit. (1 SECOND PAUSE) Nit wears his name, on his sfiirU 
Do you 'see Nit's "name on his shirt? (1 SECOND PAUSE) ' . ^ . ' 



__Gojto numeral ^f. (1 SECOND PAJ3E) Follow along by pointing to each word- 
as I read the rhyzia next to numeral 4. Ready: The elf named "Nit — sitting - 
In a pit. The picture shows Nit sitting in a pit. (■§■. SECOND PAUSE) Do you * 

m 

see Nit sitting in a pit? ({ SECOKj PAUSE) I'll read the rhyme next to 
rnuraral k again. VJhen we coma to a/i underlined word, you -read fch&t' word 
outloud. Ready: The £^ 'nar.^d (2 >SCOM» P.AUSE) is sitting in a, (2 SECOND 
> ^PAJJS5) . If you read the words Hit .xnd pit , you were., exactly right. * 

Go to numeral 5^ . (1 SECOND PAJ5E) I'll read the first sentence, then 
you read the second sentence yoursc. f. R^ady: " That's the lamp 3^1it. n Now, 
you read the second sentence outloud by yourself^ (STOP) Did you read the 
second sentence outloud? This tirce, read both sentences outloud with. me. 
Ready: "Th at's " the lanp T lit. " S;_id the 'little elf Kit . ^The picture shows . 
Kit pointing to the lamp he lit. Ytu cake an X on the l^rcp that Nit lit. 

(3 s::cc:d paus^) 

That's all for this lesson. .Gc-cdbye. : 
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In this project I have attem ted to extract, sir-ply by . 
observation, of the material:,, the objectives, skills, and'tech- 
niques in, four separate cassette- .'.Pii u^^ T JL T CI3. The rnrldei m 
employed for this analysis is a"y riation of the .presentation 
model of Tosti and ball as deline. ted in their article, "A 
behavioral Approach to Instructio. al Design and k'edia S^lectioXi' 
AV Communication Review , Vol. 17, .to; 1, Spring 1969* <** 

* 

The mr/St important par o of L axe is' tie response it* evokes 
Within the small scope of these f t ur sequences the types 'of 
response called for have be* n :r,an; ar.d varied. An^ inspection 
of the analysis will show this. 

The final test of curriculum i^. -ials i« in the empir- 
ical tryout. Likewise, in ?. syst* ..;atic approach to curriculum 
development no segment can be ;ad- ed accurately apart f^ora 
its relationship to other element: (toache-r input, ^amos , 
puzzles, and books). However, wi" hin the confines of these 
particular materials there teems * o be' a systematic approach 
derived. from science coupled with a variety and a_^eal to 
children which indicates tho afti: try of the designeer. 

These materials look vnry gor d to me, and I am looking 
forward to working in. the program. 
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S. ? Claire McCormick 
November 30, 1972 * 
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"Not until the middle of the next week, 
however, did a nontri^ial idea emerge. ... 
Suddenly I realised the potentially profound 
implications of a DNA structure in which 
the adenine residue formed hydrogen bonds 
similar to those found in crystals qf pure r 
adenine." 

James D. Watson in 
The Double Helix 



The challenge of this new age is to 
create. 

Rollo May 
November 15, 12£2 

We must learn to see everyday things 
in new ways. \^ 

Ladislaus Orsy/ 
November 18, 1972. 
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SE1CT' OM ^ 



Introduction 



This project was undertaken during the third semester of 
4 three-semester training program in curriculum development. 

The training program presented the students potential curriculum 

developers with a systems model for curriculum development. 

The model had general applications to almost any area of learning 
and instruction. The purpose of the first year of the training 
program was to enable the students to build a knowledge base for 
the production of instructional materials in a subject matter 
field or discipline of their choice. 

i 

* 

The purpose of the project presented here was to provide 
the sttflent with some prefamiliariiatton for work during a sec- 
ond year as an intern in the Reading Project at the Learning 
Research and Development Center, of the University of Pittsburgh. 

My choice of readirig as an area of concentration Stems from 
three factors: 1) my plai^for future irfiolvement "in the problems 
of urban .education in the United" States; 2) my 'perception of the 
acquisition of reading skills as one of the central problems. of • 
urban schooling; and 3) my past experience as an elementary 
school teacher and later as a teacher of Latin in secondary school. 
All of these factors provide motivation as well as some of the 
required skills. 



ERIC 



195 

109 



i 



Tha Problem 

The term problem is used here in a more general sense than 
it occurs in the literature of experimental design or in works 
devoted to the development of theory. Thie^ project addresses 
itself to what might be considered a ^kl of research in educations 
the application of scientific theory to the development of edu- 
cational .materials. This project is neither an experiment nor an 

v i 

explication of theory. It I is an analysis of educational materials 
which were produced in accordance with theoretical principles. 
If educational design is considered as an experiment, then this 
project looks at the completed experiment and analyzes it for the 
elements assumed by the theory from which the experiment origin* 
a ted. 

The structure of the New Reading System (NRS) was presented^ 

* * 

to the staff of the Learning Research and Development Center by 
^Mys. Isabel Beck f its designer, in the spring of 1972. It appears 
in written form in the document "The* Rationale "and Design of a 
Primary Grades Reading System for an Individualised Classroom" 
by Isabel L. Beck and Donna D. Mitroff. These two presentations 
represented a synthesis of theoretical knowledge of the reading 

procefs into goals and- guidelines for actual practice. 

j 

♦ 

In order to address the more general problem of theoretical 
applications to the design of educational materials, I chose a 
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rather specific task. This Usk can be described as follows: 

to analyse selected sets of response 
sheets with accompanying tape scripts 
from level 3 of for the program- 
ming techniques utilised therein. ^ » 

Since this was to be an in-dep^h study , only Your sets of ^response 
sheets. with their accompanying scripts were selected.^ I hoped 
to come to understand and to appreciate the design for Jm/which 
already existed. I needed to inter the theory which guided the 
development of these specific samples of the "materials,, and to 
understand, as far as possible within the" limitations of time 
sod expertise, how and why these materials worked, implicit 
in the task of this project was the concomitant Wk of construct- 
ing a scheme for carrying out the analysis. 



Description of NRS X 



The rfcw Reading System is Just what its name implies, a 
syatem for learning and instruction in reading. It differs fro^ 
the traditional reading programs in that it develops in a related 
snd inter-dependent manner a large ard capable-of-being-added-to 
number of components. According to Chall's study (Chall, 1967), 

4 

which was completed during the years 1962 to 1965. the -complete 
programs- included nothing more than textbooks and exercise mater- 
isls for the pupils and guidebooks for the teacher. While "sup- 
plementary" materials of many types r.ay be used with these So- 
called "complete" programs^ there is no over -all integrating 
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design. The Integra tiOn, if any, must depend upon the ingenuity 
and tlma availability of tha teacher to plan and to implement. 
The New Reading System come; to the teacher ad the artifact of. a 

carefully structured systems design. 

* • * 

Th^s^syst^ms or environmental characteristics of NRS can 
be both overwhelming and elusive to the uninitiated. The observer 
hardly knows where to begin In order to really "see? what Is there. 
Viewing the system entails looking at it "in its entirety with 

a 

/all of its ramifications f with all of its interior interactions f 
, vith all of its connections, and with full cognisance of its c 
place in its context." (Hood. in Richmond, 1970) 1 As an aid to 
understanding the structure of NRS and to placing the analysis 
within the structure of NRS. a chart highlighting the essential 
features of NRS is presented in figure #1. 

In the first two levels f those preceding the segments to 
be analysed in this paper f the most important strategy is the 
algorithm for the blending chain. "The strongest advantage pf the 
blending chain is the precisre information available to the teacher 
in terms of loca^ig an error." 2 The entire TOjU*toeaking approach 
makqs the generative principles of the language' directly available 
to the child. ' Thus the child is given an early tool for word 
attack; 
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•^Once the child enters Level III, the cassette becomes the 

' . . * 

primary vehicle for the presentation of new material. "** This 
means that during the preceding two levels self -management skills 
have had to be developed. It also means that at the begiljining of 
Level III the teacher assumes a new role, that of "traveling 
teacher". The teacher • s task now is to guide, motivate, rein* 
force and tutor. 

The strategies and conventions which formed an important 
aspect of the teacher • s role during Levels I and II are now sub- 
sumed into the cassette-led instruction. The major learning in- 
teraction shifts from the teacher present in the classroom to the 
■aster teacher made present through the design of the materials f 
the tape and response sheets, fhus a continuous colhmuni cation 
and control can replace the intermittent and less structured 

guidance of the traditional reading classroom. What h^pena at 

i \ 
Level HI is thi* implementation of a man-machine system; i.e., 

a learner, a program, and a teaching machine (the cassette tape 

recorder). 

^ jr 

At this poi^t^s^/stem conventions assume a strategic impor- 
tance. "System conventions are the arrangements of frames, the 
directions to children, and the. types of responses children must 
make. "^ It is here that both the scientific soundness and the 
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artistic inventiveness of the designer In Implementing the pre- 
scribed design decisions are evidenced. It is here that the power 
of the program is critical. The critical nature of these conven- 
tions is examined in this analysis. 

r» 

■s 

Theoretical Base \ 

This project, then, i 8 a atudy of the application of the science 
. of technology to the design and ^development of curriculum materials. * J 

Educational technology has been dWined as "the application of 
scientific knowledge .bout learning, and the conditions! of learning, 
to improve the effectiveness and efficiency of teaching and training.- 6 
Scientific knowledge about learning has been derived from labor - 
story experiments of psychologists and finds expression in various 
learning theories. This project is concerned with the contribu- 
tions of theories of learning to the design of instruction. ' ' 

f 

/ 

"V. 

Bruner' has contrasted the nature of learning theories with 
that of instructional theories.- He' identifies a theory of learning 
as descriptive and a theory of instruction as prescriptive. 
Learning theories fall into two main categories: 1) those concerned 
with controlling the stimulus or creating the environment for ' 
learning, the behavioral theories; and 2) those concerned with 
the internal processes, the cognitive theories. Theories of 
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instruction derive from theories of learning and are, because of 
the primitive stage of the development of both types of theory, 
pi-obably bept described as eclectic. / 



A theory of instruction is a normative theory in that it sets 
forth rules 8 concerning or specifying the most effective way of 
achieving knowledge or mastering^skills. A theory of instruction 
establishes a criterion and then states the conditions for meeting 
it. Theories of instruction have practical application wherever 
the teaching-learning process is in effect. 




A scientific approach to education produces procedures with 
predictable effects. It provides the kind of control of the learn- 
ing events which comes from planning and deliberate design. One 
form of planning is the specification ©f-goals and objectives. * 
Objectives identified as terminal for a course or program can be 
still further analysed. Greater specificity and detail provide 

0 

the guidance needed to construct or to choose particular instruc- 
tional items. ThV.type of objective forms a guide to the creation 

q 

of very fine-grain increases in pupil competence. 

The programmed instruction movement has this strong behavioral 
orientation. Its developments traced through Pavlov's classical 
conditioning to B". F. Skinner* s operant conditioning as well as 
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through the development of the first teaching machine by Sidney 
Pressey in the 1920 f s. Instructional programs for use in schools, 
however, are a phenomenon of only the last decade or so. 1 ^ Adapta- 
tion? of more formaf pencil and paper programs as well as the 
development of the concept of an instructional system is as yet 
a new and rare phenomenon for the professional educator. Today 
programmed learning is only gradually coming to be viewed as a 
process rather than a product.' 0 The products should developed to /\ 
facilitate the total process. 

% The purpose of programming is to maximize the rate of ♦learning, 
the length of retention as well as to enhapce the motivation of the 
student. Teaching implies first and foremost a human interaction. 12 
On the part of the teacher this interaction involves a controlling 

role f a facilitating role and a content development role. In 

i 

addition, the teacher performs roles which can be identified as 
personally responsive and positively or negatively affective. 
A carefully developed program should subsume ^fe*^ first three func- 
tions of a good teacher. 

For optimum efficiency of a program, continuous interaction 

is necessary. By designing the instruction through a master 

> 

teacher it is possible to take the planned instruction of a single 
teacher and implement it for a lar^e number of students. MMs 
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makes j^ssible the best available conditions for learning and 
instruction. This type ^^modification results in a man -machine 
system* Once such a system has been conceptualised, what is 
neceJs&ry is to optimize the man -machine functions. 



There are three critical system functions: 1) the cue faction; 
i.e. f the stimulus to which each criterion response is attached; 
2) the Motivation function, eliciting the desired performance; 
and, 3) the feedback function, "providing immediate knowledge of 
results. 1 ^ An important (characteristic of successful instructional 

.theory is that it is conceived with particular reference to the 

lk ^ 
role of student response . In applying these principles to a 

particular segment of curriculum materials, an educator needs some 

facility in determining the type of response which the materials 

have been designed to elicit. 

When an individual responds in a certain way to a given stimulus, 
that stimulus is said to control behavior. A primary objective 
of/educational technology is' the guidance of an individual's 
responses. To accomplish this, the instructor must first define 

i 

and enumerate the responses he wishes to produce. He must arragne 
stimulus conditions which will result in the desired response, 
and he must decide the response consequences. Teaching require^ 
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the establishment of successively more rigorous standards for 



$he learaer f a responses!^ thus the Sequencing of stimuli becomes, 
another Important task for the designer. 



Once the criteria or objectives of a sequence have been set, 
the construction and sequencing of frames can begin. The elements 
of a teaching frame are; stimulus, prompt, response and reinforce- 
ment. Each of these elements can be studied in detail ahd in 
relationship to one jfcnother. However, it should be kept in mind 

7 that the most important part of the frame is the response it evokes. 

** t *"*V^ Ma relif 

Frame construction is a matter of behavioral guidance and not origr 

a matter of subject matter exposition. Th< purpose of a frame is 
to let a response occur. A 

i 

The basic principles of programming can be summarised in 
these five; statements; 

1) Perform a behavioral analysis. 

2) Provide for continuous active responding' 

3) Provide for iirenediate confirmation of response * 

k) Allow for self -pacing and small steps (a relative term) . 
5) Perform a validation of the above steps. 

Ftossibly the most effective reinforcer in a programmed sequence 
Is successfully doing things which could not previously be done. 

Since the response is crucial and behavioral theory from 
which programming techniques have evolved simply surrounds the 
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response with the relevant observable eleflfents, it is necessary^ 
and profitable to resort to other theorists for a closer study 
of^the response itfSelf. Gagrie / has probably done more than 4ny 
other cognitive psychologist to analyze the types of responses 
produced in a learning situation. As an instructional psychologist, 
Gagne' has produced a scheme for the analysis of cognitive skills 
which relates behavior as closely as possible, within the limits 
of present knowledge, to the cognitive processes required to pro- 
duce the desired behavior. *7 

t 

Also related to the concept of behavioral analysis is the 
idea of "learning hierarchy". Gagne' used the term as early aq * 
1962 to refer to "a set of specified intellectual capabilities 

having, according to theoretical considerations, an ordered re- 
lationship to each other. This aspect of instructional theory 
requires the learning technologist to examine in a very detailed 
manner the type of cognitive activity which a response to a speci- 
fic frame or set of frames (segment of instructional materials) 
| elicits. The value of a learning hierarchy to the instructional 
desire* has also been well-stated by Gagne himself. "What it 
(a learning hierarchy) represents is the most probable expectation 
of greatest positive transfer for an entire sample of learners 
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concerning whom we know nothing more then what specifically r'ele^ 
rant skills they start with."** Since transfer Is the ultimate 
test of successful learning, any scheme which dfcables the designer 
• to anticipate transfer is certainly a powerful technique. 

The analysis carried out here had three parts: 1) the be- 
havioral objective hr the segment was ■ induced from, the materials; 
2) the cognitive ski^s in accordance with the scheme of Gagn. ' 
were also induced from the materials; and, 3 ) a detailed analysis 
of the elements of each frame was made in accordance with the 
model displayed in figure 2. 

What follows is a four-part application of the analysis dee. 
cribed above. Since the model developed as the project progressed, 
the first segment's analysis is somewhat different from tW 
of the next three. A discussion and conclusions follow the detailed 
Analysis. 
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HDDRL FOR EVALUATPIO RFSF0N3B 

Hua in a muortAMMio) UAjtN- 

DiO SEQUENCE FOH 



Assumptions: 



The population characteristics 
and social context are known 
to the evaluator. 



/ 



PROCEDURE 



•^Induce the behavioral objective from the 
materials. 



-Induce the cognitive skills required to 
Respond to the materials. 



"-Analyze the individual frames or groups 



of frames for: 



•Stimulus 
• Prompts 
•Response 

•Management Elements 



See model in figure 2. 



The purpose of the analysis is to 
provide the evaluator of curriculum materials 
with enough, information to determine the quality 
and intensity of learner-interaction required by- 
the materials. 
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Section VI 

4 

DISCUSSION 

" ) . 

In this project, an attempt has been zpade to extract by 
observation the objectives, the skills, and the programing tech* 
nlques Implicit or explicit in selected sets oX Instructional 
materials of NHS. the materials in question arp foujr separate 
cassette-* tape sequences from Level III of NRS. These materials 
form only one sequence and a small part of another sequence (that 
concerned with the teaching of linguistic word patterns*) from a total 
of ten such instructional sequences in the % whole of Level IH. In 

turrt, these instructional sequences form only one component in the total 

t 

system* Hence, the analysis performed here is indeed an / in-depth one* 
It deals la * very detailed manner with only a very small sector of one 
level (out of a proposed 16 levels^ of the entire reading system for . 
primary grades* ^ ^ 

Man-Machine System 

Since the concept of system in educational technology is com- 
paratively new and practically untried, there are no standard methods 
for assessing the products emerging from the application of a systems 
approach to. the design of instruction* Therefore, in order to clearly 
understand what has been produced, it' is necessary to devise some sort 
of scheme * The purpose of such a scheme is to enable the observer 

1 - 
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(examiner) to become aware of the interrelationships within the system 
and of the types and Intensities of the interactions of student with 
script and response sheet* 

Placing the script and response sheet within the context of NHS 
as a 'whole should also make the observer (examiner) aware of the inven- 
tion of a simple but potentially powerful man-machine (child-cassette) 
learning system. Such consideration should Gjnphasite the broader ap- 
plication of programmed learning which is employed when the concept 
is extended to encompass an entire system, or better a learning en- 
vironment. Also evident should be the greater possibility for diversity 
of materials which a systems approach allows. 
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scheme devised for this analysis applies only to the segments 
of cassette-led instruction not tp the system as a whole. However, it 
is necessary to have some concept of the total system in order to under- 
stand and appreciate the scheme used in the irT-depth analysis. The 
purpose of the scheme is to show how and why the materials facilitate 
learning. 

How* the Materials Work 

The materials worlf, first of all, by means of criteria or objectives 
which are derived from a very fine-grain analysis of reading behavior. 
The objectives extracted from the observation of the mater- 
ials do match the designer's objectives. 
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These objectives (see figure k ss sn example) sre simed at very small 
demonstrations of skills which in combination result 'in fluent read- 
ing behavior. The materials work by triggering student interaction 
with the voice on the tape and at the same time with the printed and 
pictorial material on the response sheets. The materials are construe- 
ted so as to provide a review or. remedial sequence for those who need it 
The student also has the option of movi^ as rapidly as he can through 
the materials since he operates the machine himself. However .most 
importantly, the materials work by making available to every student 
at the appropriate moment the expertise of a master teacher who has 
designed a rigorous strategy to control student response so as to 
produce reading skill with maximum eff ectiveness. Finally, this design 
should free the classroom teacher for personal roles of interaction 
with students in the day to day contingencies. 

. Why the Materials Work m ^ 

It is difficult to separate the "hoW^nd the "why" in speaking 
of the operations involved in the use of ^he materials. However, 
it is possible to say that the materials worFrTor these reasons: 

1) The student knows immediately whether his response 
was correct. 

2) Results are predictable from the very detailed objectives 
generated for the program. 

3) Expert task analysis and careful design and editing have 
preceded production. 
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U) Small skills are very carefully sequenced, then combined, 
and finally practiced to give fluency. 

5) Careful concideration has been given to knowledge of 
population (learner) characteristics and the type of en- 
vironment most conducive to the learning of these skills. 

Finally, the materials work by means of rigor and pay-off for the 

learner* 



The Scheme 

The scheme has been devised only for analysis of response sheets 
with 'accompanying cassette-tape. Greater emphasis has oeOn placed on 
the response sheet than on the script. The tape does, however, serve 
a vital function in the cassette-led instruction and its importance 
should be pointed up here. Research has shown that children of kinder- 

, garten age learn to read better when they have a model; i.e., these 
children benefit in a special way from listening to oral reading. 
The scheme is complete for the purpose for which it was devised. 

, However, its limitation is that it allows thef analyst to *0ook at only 
a limited segment of the system. and that it has not drawn sufficient 
attention to the oral presentation techniques on. the tapes. A much 
more complex scheme is needed to vie * the system in all of its inter - 

' relationships. 

The scheme does aerv, 3 to :;how that the purpase of the 
sludy was accomplished. Concurrent validity was .establashed 
between the Objectives and akil.a observed through the mater- 
ials and those presented "in theory by the designer-. 



9 

ERIC 



214 

128 



C0NCEU3I0NS 

A dareful analysis of t^he materials for the us*' of programming 

techniques reveals both variety of type of response as well as" 
careful sequencing in order to guide the learner to the achieve- 
ment.of the learning objective. 

Not only have scientific principles of , programming been employed 

but they have been used with the artistry that is needed to V 
produce a successful program. 

Attention has been paid not only to cognitive skills but also to 

, the affective types of responses. 

The use of the tape-cassette enables the child to learn from 

listening to a skilled reader who "can convey more meaning than 
an unskilled reader can grasp for himself even in identical words. 
The use of the tape recorder also capitalizes on the child* s 
spoken vocabulary in a special way. 
NRS gives evidence of the value of a broader concept of program- 
ming than that of programmed books. 



Possibilities for Further Study 

* Since a systems "design changes the role the classroom teacher, 

further analyses could be performed to delineate appropriate 
teacher behaviors, especially those involving the applications 
of psychological 'p^nciples and interpersonal relationships 
within the classroom setting. 
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Studies In concept fornfctlon related to the developmental level 
of the child may promde valuable information for more skillful 
instructional design. 
Studies in greater depth and extensiveness could be done on the 
utilization of systems conventions in a t^fri-machine system* 
Studies could be done to produce criteria for: relevancy of response 
"problem" worth of sequences, and probability of success of given 
sequenced * 

K 

Studies could be done to det^^lne the existence of appropriate 
practice conditions in sequences of instruction \^ita Glasgow, 
1972). . 



Concluding Comments . 

Wherp prediction becomes more precise, a correspondingly precise 
method of evaluation for decision-making becomes mandatory. One type 
of evaluation occurs during and after actual use of materials by students. 
However, another type of evaluation should occur before the materials 
are completed for production and this some type of evaluation should 
be capable of being performed by/for potential users or buyers of 
materials. Ultimately a scheme such as the one presented here might 
be perfected to serve such a function. In the' future, instructional 
technologists, engineers and analysts will be needed to perform eval- 
uative operations on instructional systems and on the materials which 
form essential components of these systems. 
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NOTES . ♦ 
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1. Richmond, W. Kenneth, ed. The Concept of Educational TeahnoloKy. 

London: Weidenfelf and Nicolson, 1970, 153- 
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of a Primary Grades Reading System for an Individualized 
Classroom . Pittsburgh; Learning Research and Develop- 
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11. Glaser, Robert. "The Psychology of Programmed Instruction" in 

International Conference on Programed Instruction. Berlin, 
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MOTES (continued) 
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and Programmed Instruction . Reading, n»» 

1965 » 19^ T 

16. ibid., 91* 
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Cartoons designed by Tommye Whiting, 
(during her internship at Frick School,) 
to test and teach observation and analysis 
of iocial interaction. The cartoons are 
part of an instructional sequence developed 
fox third^ grade social studies. 
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BIG ROY AND LITTLE JOHN 
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T. V. MADNESS" 
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THE BIG GAME 
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BROKEN WHEELS 
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Now draw in the picture and fill in what you think happened as f we1la$ what 
the three boys are saying : 

i 
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Their Products 
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STUDENTS AND THEIR PRODUCTS 

The impact of the Curriculum Design and Development Training Project can 
best be judged by its value for the students who study it, the changes it 
makes in their effectiveness as teachers and the curricula they develop. A 
brief review of the field study students and their work indicates that the 
course appeals to a broader spectrum of the population than the graduate 
education specialist for whom jLt was designed. 

Among the educators have been teachers of elementary subjects, social 
studies, math, music and fine arts, speech, English, business education, 
history, physics, earth and space science, guidance, reading, mobility for 
the blind, and special education. There have been principals, supervisors, 
school nurses, librarians. A consultant for a national vocational curriculum 
project task force, a director of biochemical technology, a superviser of 
employee development, a nursing school director, a college instructor, severa 
community college instructors, a director of business education department of 
a college, a management systems analyst, several nurses, a practical nurse 
and a dental hygienist have taken the course. Several graduate students in 
Educational Communications or other fields of education from the University 
of Pittsburgh and other universities* have been advised to take the courseP^ 
and one student from a State University is getting special permission to use 
it for the last six, credits, for her doctorate although they usually must be 
taken on the home campus, since there is nothing comparable offered at her 
.school. 

Most of the* field test students have been hard-working and very highly 
motivated by particular problems in their job situation demanding the skills 
the course teaches. Their average age is 35. Several have had doctorates, 
but on the other hand, one now enrolled does not have a bachelor's degree. 
Most students have some graduate credits. The problem which motivated one 
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high school guidance counsellor -to sign up for the course was rather unusual 
There was so much demand for it, she wanted to design a sex education course 
for parents! 

The following are some of the courses these students have developed or 
are currently developing: 

A community college course for students who need algebra for 
a degree but have a weak math background, 

* Development of* an entire curriculum for an alternate school 
for high school dropouts (by the Director), 

A liberal arts physics course with no math (sound, optics, 
electricity, heat, mechanics, etc.) 

$ 

High school oral communications. 
^ A combination of typing and remedial English for community 

college students. 

Personal development for secretarial students with emphasis 
on communication skills. 

A short story unit for eighth grade. ,f or learning disabilities 
students particularly. 

J 

A course in navigation for boaters. 
History of the Black Man 
Willing a Research Paper 

Individualization of the student-author's business education 
texts to meet her revision deadlines. 

Fundamentals of writing for New Careers students at a commun- 
ity college. 

Training courses for 1,000 prof essional and supervisory utility 

c 

company personnel. (Operating Superintendent, Supervisor of 

• * 

Division Accounting, etc.) 



2G2 

176 



Individualized course in nursing history. 

Basic course in use of library. 

Mobility training for the blind. . 
Most of these students will have impact on education beyond the courses 
they develop. One, for instance, has addressed the National Business Educator 1 
Association on the curriculum model, one has conducted in-service courses in 
objective writing in her school. One student, has worked with parochial school 
teachers. One has spoken before a national association of teachers of the 
blind. Several students are serving on curriculum committees. One student is 
taking the entire course at a very accelerated rate in order to assist in the 
changeover of all courses in her Junior College to a competency base. 

The students who took part in the field tests of the Curriculum Design 
and Development Project course have most effectively demonstrated the need, 
the value, the . effectiveness and the impact of the|model, the format and the 
total curriculum package. f 
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